[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:01 dist-test-slave-5pd3 krb5kdc[4401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:01 dist-test-slave-5pd3 krb5kdc[4401](info): set up 2 sockets
Sep 27 22:57:01 dist-test-slave-5pd3 krb5kdc[4401](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260927 22:57:02.990213 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.002s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:03 dist-test-slave-5pd3 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:57:03.230206 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:43447
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:43447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:03.662062 4414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:03.662456 4414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:03.662569 4414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:03.671414 4414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:03.671695 4414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:03.671780 4414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:03.671850 4414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:03.671901 4414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:03.684146 4414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:43447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:43447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:03.685981 4414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:03.687992 4414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:03.703437 4419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:03.703390 4420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:03.703358 4422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:03.704486 4414 server_base.cc:1034] running on GCE node
I20260927 22:57:03.705444 4414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:03.707358 4414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:03.708770 4414 hybrid_clock.cc:648] HybridClock initialized: now 1790549823708643 us; error 114 us; skew 500 ppm
I20260927 22:57:03.709317 4414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:03 dist-test-slave-5pd3 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549823, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:03.713375 4414 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:57:03.715417 4414 webserver.cc:460] Webserver started at http://127.4.74.190:37983/ using document root <none> and password file <none>
I20260927 22:57:03.716383 4414 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:03.716533 4414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:03.716952 4414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:03.720827 4414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "49a32c053b8649e6b8ab9ff6063896b7"
format_stamp: "Formatted at 2026-09-27 22:57:03 on dist-test-slave-5pd3"
server_key: "ba3b9311d6111eff4edb5bb686e5e8e3"
server_key_iv: "7d4c0a7f453da25f600d0f617100c0a0"
server_key_version: "encryptionkey@0"
I20260927 22:57:03.721778 4414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "49a32c053b8649e6b8ab9ff6063896b7"
format_stamp: "Formatted at 2026-09-27 22:57:03 on dist-test-slave-5pd3"
server_key: "ba3b9311d6111eff4edb5bb686e5e8e3"
server_key_iv: "7d4c0a7f453da25f600d0f617100c0a0"
server_key_version: "encryptionkey@0"
I20260927 22:57:03.729014 4414 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260927 22:57:03.734557 4429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:03.736685 4414 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260927 22:57:03.736963 4414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "49a32c053b8649e6b8ab9ff6063896b7"
format_stamp: "Formatted at 2026-09-27 22:57:03 on dist-test-slave-5pd3"
server_key: "ba3b9311d6111eff4edb5bb686e5e8e3"
server_key_iv: "7d4c0a7f453da25f600d0f617100c0a0"
server_key_version: "encryptionkey@0"
I20260927 22:57:03.737218 4414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:03.761476 4414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:03.767037 4414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:03.767424 4414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:03.792289 4414 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:43447
I20260927 22:57:03.792333 4480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:43447 every 8 connection(s)
I20260927 22:57:03.794386 4414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:03.799829 4481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:03.803057 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 4414
I20260927 22:57:03.803442 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:03.805753 4394 external_mini_cluster.cc:1428] Setting key 9011b93bfc3b34d564f1719caccfc2c9
I20260927 22:57:03.813235 4481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7: Bootstrap starting.
I20260927 22:57:03.817341 4481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:03.818643 4481 log.cc:824] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:03.822190 4481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7: No bootstrap required, opened a new log
I20260927 22:57:03.833397 4481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a32c053b8649e6b8ab9ff6063896b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43447 } }
I20260927 22:57:03.833998 4481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:03.834165 4481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a32c053b8649e6b8ab9ff6063896b7, State: Initialized, Role: FOLLOWER
I20260927 22:57:03.834952 4481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a32c053b8649e6b8ab9ff6063896b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43447 } }
I20260927 22:57:03.835242 4481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:03.835402 4481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:03.835635 4481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:03.839119 4481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a32c053b8649e6b8ab9ff6063896b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43447 } }
I20260927 22:57:03.839883 4481 leader_election.cc:302] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49a32c053b8649e6b8ab9ff6063896b7; no voters:
I20260927 22:57:03.840762 4481 leader_election.cc:288] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:03.841185 4486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:03.842617 4486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [term 1 LEADER]: Becoming Leader. State: Replica: 49a32c053b8649e6b8ab9ff6063896b7, State: Running, Role: LEADER
I20260927 22:57:03.843997 4486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a32c053b8649e6b8ab9ff6063896b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43447 } }
I20260927 22:57:03.844391 4481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [sys.catalog]: configured and running, proceeding with master startup.
Sep 27 22:57:03 dist-test-slave-5pd3 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:03.848254 4487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49a32c053b8649e6b8ab9ff6063896b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a32c053b8649e6b8ab9ff6063896b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43447 } } }
I20260927 22:57:03.848796 4487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:03.849419 4488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49a32c053b8649e6b8ab9ff6063896b7. Latest consensus state: current_term: 1 leader_uuid: "49a32c053b8649e6b8ab9ff6063896b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49a32c053b8649e6b8ab9ff6063896b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43447 } } }
I20260927 22:57:03.849748 4488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:03.856276 4491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:03.865715 4491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:03.869707 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:03.810984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56808 (local address 127.4.74.190:43447)
0927 22:57:03.812194 (+ 1210us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:03.812219 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:03.813847 (+ 1628us) server_negotiation.cc:407] Connection header received
0927 22:57:03.825168 (+ 11321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:03.825208 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:03.825899 (+ 691us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:03.826519 (+ 620us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:03.831384 (+ 4865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:03.833760 (+ 2376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:03.836637 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:03.838024 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:03.838291 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:03.850963 (+ 12672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:03.851000 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:03.851034 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:03.851130 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:03.858605 (+ 7475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:03.860146 (+ 1541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:03.860162 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:03.860179 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:03.860582 (+ 403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:03.861419 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:03.861426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:03.861431 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:03.863210 (+ 1779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:03.866468 (+ 3258us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:03.866899 (+ 431us) server_negotiation.cc:299] Negotiation successful
0927 22:57:03.867717 (+ 818us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":556,"threads_started":1}
I20260927 22:57:03.885790 4491 catalog_manager.cc:1347] Generated new cluster ID: 020251a2d885456c881d20fb98a885cb
I20260927 22:57:03.885987 4491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:03.898448 4491 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:03.900840 4491 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:03.917037 4491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49a32c053b8649e6b8ab9ff6063896b7: Generated new TSK 0
I20260927 22:57:03.918630 4491 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:57:03.972932 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43447
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:04.340641 4508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:04.340937 4508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:04.341101 4508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:04.348753 4508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:04.348875 4508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:04.349058 4508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:57:04.359606 4508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43447
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:04.360984 4508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:04.362412 4508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:04.377664 4514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:04.377669 4513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:04.377610 4516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:04.378854 4508 server_base.cc:1034] running on GCE node
I20260927 22:57:04.379765 4508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:04.380990 4508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:04.382246 4508 hybrid_clock.cc:648] HybridClock initialized: now 1790549824382144 us; error 78 us; skew 500 ppm
I20260927 22:57:04.382604 4508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:04 dist-test-slave-5pd3 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549824, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:04.386659 4508 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:57:04.388656 4508 webserver.cc:460] Webserver started at http://127.4.74.129:33991/ using document root <none> and password file <none>
I20260927 22:57:04.389598 4508 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:04.389724 4508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:04.390062 4508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:04.394008 4508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "8def9b3fc24745ea8dda9f2f6b6cda1f"
format_stamp: "Formatted at 2026-09-27 22:57:04 on dist-test-slave-5pd3"
server_key: "205837f0eaac30002ee88f48b0ffcd29"
server_key_iv: "3b028b9ddc029fe412864deefcd8b475"
server_key_version: "encryptionkey@0"
I20260927 22:57:04.394986 4508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "8def9b3fc24745ea8dda9f2f6b6cda1f"
format_stamp: "Formatted at 2026-09-27 22:57:04 on dist-test-slave-5pd3"
server_key: "205837f0eaac30002ee88f48b0ffcd29"
server_key_iv: "3b028b9ddc029fe412864deefcd8b475"
server_key_version: "encryptionkey@0"
I20260927 22:57:04.401888 4508 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260927 22:57:04.406997 4523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:04.409277 4508 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260927 22:57:04.409657 4508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "8def9b3fc24745ea8dda9f2f6b6cda1f"
format_stamp: "Formatted at 2026-09-27 22:57:04 on dist-test-slave-5pd3"
server_key: "205837f0eaac30002ee88f48b0ffcd29"
server_key_iv: "3b028b9ddc029fe412864deefcd8b475"
server_key_version: "encryptionkey@0"
I20260927 22:57:04.409996 4508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:04.450342 4508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:04.455651 4508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:04.456024 4508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:04.457276 4508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:04.459178 4508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:04.459301 4508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:04.459414 4508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:04.459463 4508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:04.500524 4508 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:41177
I20260927 22:57:04.500626 4635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:41177 every 8 connection(s)
I20260927 22:57:04.502192 4508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:57:04.503803 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 4508
I20260927 22:57:04.504056 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:57:04.504848 4394 external_mini_cluster.cc:1428] Setting key 0a721ddac0861a2a04c2a5629ad5e703
Sep 27 22:57:04 dist-test-slave-5pd3 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549824, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
I20260927 22:57:04.529402 4638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:04.505925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:54039 (local address 127.4.74.190:43447)
0927 22:57:04.506512 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:04.506518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:04.508088 (+ 1570us) server_negotiation.cc:407] Connection header received
0927 22:57:04.510153 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:04.510158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:04.510263 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:04.510384 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:04.512097 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:04.513302 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:04.515591 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:04.516368 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:04.516539 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:04.521047 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:04.521054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:04.521060 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:04.521109 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:04.524622 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:04.525974 (+ 1352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:04.525985 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:04.525992 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:04.526246 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:04.527031 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:04.527036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:04.527038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:04.527399 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:04.527619 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:04.528709 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0927 22:57:04.529055 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":320,"threads_started":1}
I20260927 22:57:04.530887 4639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:04.506822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43447 (local address 127.4.74.129:54039)
0927 22:57:04.507920 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:04.507969 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:04.509859 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:04.510652 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:04.510661 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:04.511234 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:04.511923 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:04.511941 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:04.513599 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:04.513605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:04.515363 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:04.515386 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:04.516711 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:04.516718 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:04.516988 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:04.517913 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:04.517943 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:04.520720 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:04.524970 (+ 4250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:04.524980 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:04.524997 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:04.525725 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:04.526485 (+ 760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:04.526491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:04.526496 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:04.526823 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:04.527594 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:04.527606 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:04.528193 (+ 587us) client_negotiation.cc:769] Sending connection context
0927 22:57:04.528582 (+ 389us) client_negotiation.cc:240] Negotiation successful
0927 22:57:04.529269 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":422,"threads_started":1}
I20260927 22:57:04.533043 4636 heartbeater.cc:347] Connected to a master server at 127.4.74.190:43447
I20260927 22:57:04.533406 4636 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:04.534531 4636 heartbeater.cc:511] Master 127.4.74.190:43447 requested a full tablet report, sending...
I20260927 22:57:04.537247 4446 ts_manager.cc:194] Registered new tserver with Master: 8def9b3fc24745ea8dda9f2f6b6cda1f (127.4.74.129:41177)
I20260927 22:57:04.538923 4446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:54039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:57:04.551896 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43447
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:04.955273 4642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:04.955695 4642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:04.955875 4642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:04.963538 4642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:04.963770 4642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:04.963959 4642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:57:04.974352 4642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43447
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:04.975826 4642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:04.977291 4642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:04.993198 4647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:04.993271 4650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:04.993258 4648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:04.995060 4642 server_base.cc:1034] running on GCE node
I20260927 22:57:04.996047 4642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:04.997406 4642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:04.998732 4642 hybrid_clock.cc:648] HybridClock initialized: now 1790549824998668 us; error 70 us; skew 500 ppm
I20260927 22:57:04.999207 4642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:04 dist-test-slave-5pd3 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549824, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:05.003185 4642 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:57:05.005332 4642 webserver.cc:460] Webserver started at http://127.4.74.130:46829/ using document root <none> and password file <none>
I20260927 22:57:05.006259 4642 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:05.006430 4642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:05.006835 4642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:05.010761 4642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "ac13193cdfa640ca9a487d8f7844386d"
format_stamp: "Formatted at 2026-09-27 22:57:05 on dist-test-slave-5pd3"
server_key: "ef67135d080bc07914d3542b12e3540b"
server_key_iv: "88093a357c0979d2430eba441071f0a8"
server_key_version: "encryptionkey@0"
I20260927 22:57:05.011888 4642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "ac13193cdfa640ca9a487d8f7844386d"
format_stamp: "Formatted at 2026-09-27 22:57:05 on dist-test-slave-5pd3"
server_key: "ef67135d080bc07914d3542b12e3540b"
server_key_iv: "88093a357c0979d2430eba441071f0a8"
server_key_version: "encryptionkey@0"
I20260927 22:57:05.018990 4642 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260927 22:57:05.024156 4657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:05.026201 4642 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.004s
I20260927 22:57:05.026481 4642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "ac13193cdfa640ca9a487d8f7844386d"
format_stamp: "Formatted at 2026-09-27 22:57:05 on dist-test-slave-5pd3"
server_key: "ef67135d080bc07914d3542b12e3540b"
server_key_iv: "88093a357c0979d2430eba441071f0a8"
server_key_version: "encryptionkey@0"
I20260927 22:57:05.026749 4642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:05.045596 4642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:05.050763 4642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:05.051110 4642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:05.052386 4642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:05.054225 4642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:05.054363 4642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:05.054538 4642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:05.054628 4642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:05.095636 4642 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:42079
I20260927 22:57:05.095745 4769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:42079 every 8 connection(s)
I20260927 22:57:05.097287 4642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:57:05.104833 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 4642
I20260927 22:57:05.105098 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:57:05.105775 4394 external_mini_cluster.cc:1428] Setting key c54d39772221ea533ef97e0138c97e21
Sep 27 22:57:05 dist-test-slave-5pd3 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549824, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 22:57:05.128189 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:05.100593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:58211 (local address 127.4.74.190:43447)
0927 22:57:05.101136 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:05.101142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:05.102383 (+ 1241us) server_negotiation.cc:407] Connection header received
0927 22:57:05.105387 (+ 3004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:05.105392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:05.105484 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:05.105626 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:05.108165 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.109144 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.113190 (+ 4046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.113689 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.113923 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:05.119466 (+ 5543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:05.119477 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:05.119485 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:05.119569 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:05.123754 (+ 4185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.124850 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.124856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.124860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.125049 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.125660 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.125667 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.125670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.126112 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:05.126297 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:05.127338 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0927 22:57:05.127678 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":302,"threads_started":1}
I20260927 22:57:05.129549 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:05.101149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43447 (local address 127.4.74.130:58211)
0927 22:57:05.102125 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:05.102190 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:05.104902 (+ 2712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:05.106046 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:05.106063 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:05.106971 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:05.107935 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:05.107961 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.109425 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.109435 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:05.112810 (+ 3375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:05.112850 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.114095 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.114107 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:05.114601 (+ 494us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:05.115748 (+ 1147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:05.115791 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:05.119058 (+ 3267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:05.123993 (+ 4935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:05.124004 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:05.124024 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:05.124636 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:05.125213 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:05.125226 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:05.125229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:05.125466 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:05.126285 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:05.126294 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:05.126778 (+ 484us) client_negotiation.cc:769] Sending connection context
0927 22:57:05.127154 (+ 376us) client_negotiation.cc:240] Negotiation successful
0927 22:57:05.127861 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":397,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
I20260927 22:57:05.131979 4770 heartbeater.cc:347] Connected to a master server at 127.4.74.190:43447
I20260927 22:57:05.132484 4770 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:05.133801 4770 heartbeater.cc:511] Master 127.4.74.190:43447 requested a full tablet report, sending...
I20260927 22:57:05.136219 4446 ts_manager.cc:194] Registered new tserver with Master: ac13193cdfa640ca9a487d8f7844386d (127.4.74.130:42079)
I20260927 22:57:05.137423 4446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:58211
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:57:05.156903 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43447
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:05.509927 4776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:05.510224 4776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:05.510381 4776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:05.517964 4776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:05.518096 4776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:05.518282 4776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:57:05.528577 4776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43447
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:05.529924 4776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:05.531309 4776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 22:57:05.542469 4636 heartbeater.cc:503] Master 127.4.74.190:43447 was elected leader, sending a full tablet report...
W20260927 22:57:05.547963 4781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:05.548018 4784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:05.548053 4782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:05.550132 4776 server_base.cc:1034] running on GCE node
I20260927 22:57:05.550858 4776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:05.552042 4776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:05.553285 4776 hybrid_clock.cc:648] HybridClock initialized: now 1790549825553207 us; error 65 us; skew 500 ppm
I20260927 22:57:05.553707 4776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:05 dist-test-slave-5pd3 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549825, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:05.557533 4776 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:57:05.559535 4776 webserver.cc:460] Webserver started at http://127.4.74.131:39899/ using document root <none> and password file <none>
I20260927 22:57:05.560374 4776 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:05.560542 4776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:05.560890 4776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:05.564179 4776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "695a2837c9bc467d981c6c02eac67192"
format_stamp: "Formatted at 2026-09-27 22:57:05 on dist-test-slave-5pd3"
server_key: "fdc1dc702773d65649e964b6dc24839d"
server_key_iv: "230ef29325a098cb5b8e9767910562f1"
server_key_version: "encryptionkey@0"
I20260927 22:57:05.565088 4776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "695a2837c9bc467d981c6c02eac67192"
format_stamp: "Formatted at 2026-09-27 22:57:05 on dist-test-slave-5pd3"
server_key: "fdc1dc702773d65649e964b6dc24839d"
server_key_iv: "230ef29325a098cb5b8e9767910562f1"
server_key_version: "encryptionkey@0"
I20260927 22:57:05.571365 4776 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260927 22:57:05.576043 4791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:05.578004 4776 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:57:05.578279 4776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "695a2837c9bc467d981c6c02eac67192"
format_stamp: "Formatted at 2026-09-27 22:57:05 on dist-test-slave-5pd3"
server_key: "fdc1dc702773d65649e964b6dc24839d"
server_key_iv: "230ef29325a098cb5b8e9767910562f1"
server_key_version: "encryptionkey@0"
I20260927 22:57:05.578538 4776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:05.600338 4776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:05.604972 4776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:05.605295 4776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:05.606407 4776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:05.607992 4776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:05.608099 4776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:05.608260 4776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:05.608335 4776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:05.648082 4776 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:34405
I20260927 22:57:05.648186 4903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:34405 every 8 connection(s)
I20260927 22:57:05.649535 4776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:57:05.658726 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 4776
I20260927 22:57:05.658994 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:57:05.659700 4394 external_mini_cluster.cc:1428] Setting key d7ebf65a0d59fc7c63c34e9cf60ea9b7
Sep 27 22:57:05 dist-test-slave-5pd3 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549825, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:05.676721 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:05.652969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:48721 (local address 127.4.74.190:43447)
0927 22:57:05.653577 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:05.653582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:05.654641 (+ 1059us) server_negotiation.cc:407] Connection header received
0927 22:57:05.657019 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:05.657025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:05.657152 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:05.657288 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:05.660023 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.661031 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.663442 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.664223 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.664397 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:05.669083 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:05.669089 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:05.669093 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:05.669146 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:05.672325 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.673463 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.673467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.673470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.673619 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.674172 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.674176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.674178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.674551 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:05.674689 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:05.675931 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0927 22:57:05.676207 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":328,"threads_started":1}
I20260927 22:57:05.678341 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:05.653372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43447 (local address 127.4.74.131:48721)
0927 22:57:05.654315 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:05.654377 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:05.656622 (+ 2245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:05.657648 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:05.657659 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:05.658477 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:05.659688 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:05.659725 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.661210 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.661216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:05.663251 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:05.663267 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.664579 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.664588 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:05.664945 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:05.665945 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:05.665974 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:05.668720 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:05.672588 (+ 3868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:05.672599 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:05.672619 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:05.673288 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:05.673734 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:05.673737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:05.673740 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:05.674004 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:05.674731 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:05.674739 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:05.675375 (+ 636us) client_negotiation.cc:769] Sending connection context
0927 22:57:05.675804 (+ 429us) client_negotiation.cc:240] Negotiation successful
0927 22:57:05.676538 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":369,"threads_started":1}
I20260927 22:57:05.680101 4904 heartbeater.cc:347] Connected to a master server at 127.4.74.190:43447
I20260927 22:57:05.680550 4904 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:05.681394 4904 heartbeater.cc:511] Master 127.4.74.190:43447 requested a full tablet report, sending...
I20260927 22:57:05.683092 4446 ts_manager.cc:194] Registered new tserver with Master: 695a2837c9bc467d981c6c02eac67192 (127.4.74.131:34405)
I20260927 22:57:05.684074 4446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:48721
I20260927 22:57:05.691421 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 22:57:05 dist-test-slave-5pd3 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549825, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 22:57:05 dist-test-slave-5pd3 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549825, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:05.744794 4912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:05.725623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35572 (local address 127.4.74.129:41177)
0927 22:57:05.726698 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:05.726707 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:05.726742 (+ 35us) server_negotiation.cc:407] Connection header received
0927 22:57:05.726838 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:05.726846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:05.727143 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:05.727331 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:05.729003 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.730147 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.731903 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.732864 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.732984 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:05.737448 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:05.737462 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:05.737480 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:05.737528 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:05.741133 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.742035 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.742043 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.742046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.742207 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.742888 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.742893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.742896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.743343 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:05.743742 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:05.744051 (+ 309us) server_negotiation.cc:299] Negotiation successful
0927 22:57:05.744344 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":891,"thread_start_us":633,"threads_started":1}
W20260927 22:57:05.746444 4551 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35572
Sep 27 22:57:05 dist-test-slave-5pd3 krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 22:57:05 dist-test-slave-5pd3 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:05.797730 4912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:05.780893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35580 (local address 127.4.74.129:41177)
0927 22:57:05.781236 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:05.781242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:05.781478 (+ 236us) server_negotiation.cc:407] Connection header received
0927 22:57:05.781633 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:05.781638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:05.781745 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:05.781910 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:05.783775 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.784672 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.786082 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.786564 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.786677 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:05.791648 (+ 4971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:05.791655 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:05.791659 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:05.791704 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:05.794724 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.795536 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.795573 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.795578 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.795741 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.796154 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.796161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.796164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.796457 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:05.796725 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:05.797098 (+ 373us) server_negotiation.cc:299] Negotiation successful
0927 22:57:05.797351 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260927 22:57:05.799609 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 4508
I20260927 22:57:05.815062 4912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:05.800762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.4.74.129:41177)
0927 22:57:05.800941 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:05.800946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:05.801434 (+ 488us) server_negotiation.cc:407] Connection header received
0927 22:57:05.801592 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:05.801596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:05.801659 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:05.801770 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:05.803271 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.804072 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.805629 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.806047 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.806159 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:05.808996 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:05.809001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:05.809005 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:05.809044 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:05.811874 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.812790 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.812794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.812797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.812950 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.813360 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.813365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.813367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.813663 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:05.813871 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:05.814348 (+ 477us) server_negotiation.cc:299] Negotiation successful
0927 22:57:05.814622 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 22:57:05.816155 4631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:05.905216 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 4508
I20260927 22:57:05.928895 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 4642
Sep 27 22:57:05 dist-test-slave-5pd3 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:57:05.949558 4919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:05.929849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33176 (local address 127.4.74.130:42079)
0927 22:57:05.930503 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:05.930511 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:05.930542 (+ 31us) server_negotiation.cc:407] Connection header received
0927 22:57:05.930613 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:05.930620 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:05.930852 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:05.931041 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:05.932685 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.933872 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.935353 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:05.936098 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:05.936229 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:05.941509 (+ 5280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:05.941530 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:05.941549 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:05.941596 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:05.945673 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.946862 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.946872 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.946874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.947073 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:05.947731 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:05.947736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:05.947739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:05.948180 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:05.948438 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:05.948844 (+ 406us) server_negotiation.cc:299] Negotiation successful
0927 22:57:05.949092 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":360,"threads_started":1}
I20260927 22:57:05.951123 4765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:06.046427 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 4642
I20260927 22:57:06.063459 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 4776
Sep 27 22:57:06 dist-test-slave-5pd3 krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:57:06.083281 4921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:06.064803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58520 (local address 127.4.74.131:34405)
0927 22:57:06.065538 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:06.065547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:06.065580 (+ 33us) server_negotiation.cc:407] Connection header received
0927 22:57:06.065651 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:06.065659 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:06.065931 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:06.066089 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:06.067686 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:06.068862 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:06.070330 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:06.070998 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:06.071119 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:06.076182 (+ 5063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:06.076195 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:06.076214 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:06.076266 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:06.079722 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:06.080696 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:06.080711 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:06.080713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:06.080873 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:06.081282 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:06.081288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:06.081291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:06.081699 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:06.082076 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:06.082407 (+ 331us) server_negotiation.cc:299] Negotiation successful
0927 22:57:06.082734 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":374,"threads_started":1}
I20260927 22:57:06.084960 4899 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:06.183208 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 4776
I20260927 22:57:06.200569 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 4414
I20260927 22:57:06.202162 4476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:06.326725 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 4414
2026-09-27T22:57:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4531 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:06 dist-test-slave-5pd3 krb5kdc[4926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:06 dist-test-slave-5pd3 krb5kdc[4926](info): set up 2 sockets
Sep 27 22:57:06 dist-test-slave-5pd3 krb5kdc[4926](info): commencing operation
krb5kdc: starting...
W20260927 22:57:07.447942 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:07 dist-test-slave-5pd3 krb5kdc[4926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:57:07.660264 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:38075
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:38075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:07.993654 4939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:07.994061 4939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:07.994859 4939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:08.003062 4939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:08.003264 4939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:08.003387 4939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:08.003489 4939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:08.003598 4939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:08.014076 4939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:38075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:38075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:08.015592 4939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:08.017030 4939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:08.031076 4944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:08.031040 4945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:08.031126 4947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:08.032224 4939 server_base.cc:1034] running on GCE node
I20260927 22:57:08.033200 4939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:08.034852 4939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:08.036111 4939 hybrid_clock.cc:648] HybridClock initialized: now 1790549828036060 us; error 50 us; skew 500 ppm
I20260927 22:57:08.036543 4939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:08 dist-test-slave-5pd3 krb5kdc[4926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549828, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:08.045985 4939 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:57:08.048193 4939 webserver.cc:460] Webserver started at http://127.4.74.190:36311/ using document root <none> and password file <none>
I20260927 22:57:08.049010 4939 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:08.049304 4939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:08.049675 4939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:08.052920 4939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "3122145b670f48a4a588581ed4728d55"
format_stamp: "Formatted at 2026-09-27 22:57:08 on dist-test-slave-5pd3"
server_key: "e42d73912bdae11a2dbfb4a01d4a42b8"
server_key_iv: "80b1c8e22b918d33b07ee175ae740b23"
server_key_version: "encryptionkey@0"
I20260927 22:57:08.053732 4939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "3122145b670f48a4a588581ed4728d55"
format_stamp: "Formatted at 2026-09-27 22:57:08 on dist-test-slave-5pd3"
server_key: "e42d73912bdae11a2dbfb4a01d4a42b8"
server_key_iv: "80b1c8e22b918d33b07ee175ae740b23"
server_key_version: "encryptionkey@0"
I20260927 22:57:08.059775 4939 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260927 22:57:08.063845 4954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:08.065486 4939 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:57:08.065747 4939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "3122145b670f48a4a588581ed4728d55"
format_stamp: "Formatted at 2026-09-27 22:57:08 on dist-test-slave-5pd3"
server_key: "e42d73912bdae11a2dbfb4a01d4a42b8"
server_key_iv: "80b1c8e22b918d33b07ee175ae740b23"
server_key_version: "encryptionkey@0"
I20260927 22:57:08.065979 4939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:08.082252 4939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:08.087074 4939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:08.087428 4939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:08.115381 4939 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:38075
I20260927 22:57:08.115432 5005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:38075 every 8 connection(s)
I20260927 22:57:08.117115 4939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:08.122275 5006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:08.123093 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 4939
I20260927 22:57:08.123368 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:08.124161 4394 external_mini_cluster.cc:1428] Setting key ce0759bb01f0cb3007959e8a37606892
I20260927 22:57:08.139711 5006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55: Bootstrap starting.
Sep 27 22:57:08 dist-test-slave-5pd3 krb5kdc[4926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:08.144434 5006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:08.146262 5006 log.cc:824] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:08.150591 5006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55: No bootstrap required, opened a new log
I20260927 22:57:08.153775 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:08.126866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41782 (local address 127.4.74.190:38075)
0927 22:57:08.127889 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:08.127907 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:08.127955 (+ 48us) server_negotiation.cc:407] Connection header received
0927 22:57:08.130050 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:08.130091 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:08.130585 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:08.130996 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:08.133199 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:08.134650 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:08.136812 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:08.137599 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:08.137757 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:08.143738 (+ 5981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:08.143748 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:08.143769 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:08.143823 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:08.148197 (+ 4374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:08.149154 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:08.149167 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:08.149182 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:08.149434 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:08.149868 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:08.149874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:08.149876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:08.150726 (+ 850us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:08.151050 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:08.151640 (+ 590us) server_negotiation.cc:299] Negotiation successful
0927 22:57:08.152137 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":442,"threads_started":1}
I20260927 22:57:08.161504 5006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3122145b670f48a4a588581ed4728d55" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 38075 } }
I20260927 22:57:08.162863 5006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:08.163005 5006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3122145b670f48a4a588581ed4728d55, State: Initialized, Role: FOLLOWER
I20260927 22:57:08.163887 5006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3122145b670f48a4a588581ed4728d55" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 38075 } }
I20260927 22:57:08.164181 5006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:08.164307 5006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:08.164472 5006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:08.167775 5006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3122145b670f48a4a588581ed4728d55" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 38075 } }
I20260927 22:57:08.168522 5006 leader_election.cc:302] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3122145b670f48a4a588581ed4728d55; no voters:
I20260927 22:57:08.169494 5006 leader_election.cc:288] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:08.169699 5011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:08.170853 5011 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [term 1 LEADER]: Becoming Leader. State: Replica: 3122145b670f48a4a588581ed4728d55, State: Running, Role: LEADER
I20260927 22:57:08.171598 5011 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3122145b670f48a4a588581ed4728d55" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 38075 } }
I20260927 22:57:08.172618 5006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:08.175328 5012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3122145b670f48a4a588581ed4728d55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3122145b670f48a4a588581ed4728d55" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 38075 } } }
I20260927 22:57:08.175786 5012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:08.176266 5013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3122145b670f48a4a588581ed4728d55. Latest consensus state: current_term: 1 leader_uuid: "3122145b670f48a4a588581ed4728d55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3122145b670f48a4a588581ed4728d55" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 38075 } } }
I20260927 22:57:08.176566 5013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:08.178489 5016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:08.184490 5016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:08.195655 5016 catalog_manager.cc:1347] Generated new cluster ID: 7c9b505f7b2d4e3c93db7471442e064c
I20260927 22:57:08.195802 5016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:08.213557 5016 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:08.215124 5016 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:08.233654 5016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3122145b670f48a4a588581ed4728d55: Generated new TSK 0
I20260927 22:57:08.234620 5016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:57:08.287431 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:38075
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:08.638845 5032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:08.639156 5032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:08.639257 5032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:08.646970 5032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:08.647107 5032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:08.647233 5032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:57:08.658680 5032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:38075
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:08.660063 5032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:08.661669 5032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:08.676800 5038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:08.677304 5037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:08.678102 5040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:08.679997 5032 server_base.cc:1034] running on GCE node
I20260927 22:57:08.680809 5032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:08.681958 5032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:08.683270 5032 hybrid_clock.cc:648] HybridClock initialized: now 1790549828683205 us; error 63 us; skew 500 ppm
I20260927 22:57:08.683727 5032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:08 dist-test-slave-5pd3 krb5kdc[4926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549828, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:08.692327 5032 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:57:08.694149 5032 webserver.cc:460] Webserver started at http://127.4.74.129:36233/ using document root <none> and password file <none>
I20260927 22:57:08.694907 5032 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:08.695013 5032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:08.695384 5032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:08.698523 5032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "954b57b2f9324e0ab300279ae6fcc33d"
format_stamp: "Formatted at 2026-09-27 22:57:08 on dist-test-slave-5pd3"
server_key: "68d5dd529e026755414bda9ff7908e35"
server_key_iv: "f0397e4f14553dc9beff78de4b95605d"
server_key_version: "encryptionkey@0"
I20260927 22:57:08.699378 5032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "954b57b2f9324e0ab300279ae6fcc33d"
format_stamp: "Formatted at 2026-09-27 22:57:08 on dist-test-slave-5pd3"
server_key: "68d5dd529e026755414bda9ff7908e35"
server_key_iv: "f0397e4f14553dc9beff78de4b95605d"
server_key_version: "encryptionkey@0"
I20260927 22:57:08.705295 5032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260927 22:57:08.709666 5047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:08.711369 5032 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260927 22:57:08.711627 5032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "954b57b2f9324e0ab300279ae6fcc33d"
format_stamp: "Formatted at 2026-09-27 22:57:08 on dist-test-slave-5pd3"
server_key: "68d5dd529e026755414bda9ff7908e35"
server_key_iv: "f0397e4f14553dc9beff78de4b95605d"
server_key_version: "encryptionkey@0"
I20260927 22:57:08.711862 5032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:08.741788 5032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:08.746277 5032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:08.746618 5032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:08.747794 5032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:08.749254 5032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:08.749331 5032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:08.749509 5032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:08.749575 5032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:08.789426 5032 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:41987
I20260927 22:57:08.789534 5159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:41987 every 8 connection(s)
I20260927 22:57:08.790820 5032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:57:08.798262 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 5032
I20260927 22:57:08.798521 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:57:08.799160 4394 external_mini_cluster.cc:1428] Setting key 42fff778b4284d7f6b61f0b5ddbaa41f
Sep 27 22:57:08 dist-test-slave-5pd3 krb5kdc[4926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549828, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:08.819348 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:08.794246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:56069 (local address 127.4.74.190:38075)
0927 22:57:08.794818 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:08.794824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:08.795824 (+ 1000us) server_negotiation.cc:407] Connection header received
0927 22:57:08.798070 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:08.798076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:08.798190 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:08.798325 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:08.800662 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:08.801720 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:08.803728 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:08.804485 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:08.804685 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:08.809971 (+ 5286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:08.809978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:08.809981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:08.810019 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:08.814531 (+ 4512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:08.816169 (+ 1638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:08.816174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:08.816178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:08.816359 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:08.817040 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:08.817045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:08.817049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:08.817459 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:08.817633 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:08.818739 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0927 22:57:08.818975 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":319,"threads_started":1}
I20260927 22:57:08.821112 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:08.794714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:38075 (local address 127.4.74.129:56069)
0927 22:57:08.795607 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:08.795679 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:08.797685 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:08.798653 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:08.798668 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:08.799412 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:08.800364 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:08.800388 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:08.801933 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:08.801938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:08.803451 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:08.803466 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:08.804826 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:08.804833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:08.805097 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:08.806097 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:08.806141 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:08.809664 (+ 3523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:08.814838 (+ 5174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:08.814851 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:08.814872 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:08.815796 (+ 924us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:08.816520 (+ 724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:08.816524 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:08.816531 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:08.816810 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:08.817608 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:08.817617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:08.818202 (+ 585us) client_negotiation.cc:769] Sending connection context
0927 22:57:08.818587 (+ 385us) client_negotiation.cc:240] Negotiation successful
0927 22:57:08.819184 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":324,"threads_started":1}
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 22:57:08.823900 5160 heartbeater.cc:347] Connected to a master server at 127.4.74.190:38075
I20260927 22:57:08.824429 5160 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
I20260927 22:57:08.826256 5160 heartbeater.cc:511] Master 127.4.74.190:38075 requested a full tablet report, sending...
I20260927 22:57:08.832475 4971 ts_manager.cc:194] Registered new tserver with Master: 954b57b2f9324e0ab300279ae6fcc33d (127.4.74.129:41987)
I20260927 22:57:08.835027 4971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:56069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:57:08.857092 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:38075
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:09.229154 5166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:09.229465 5166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:09.229562 5166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:09.237495 5166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:09.237665 5166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:09.237787 5166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:57:09.248023 5166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:38075
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:09.249351 5166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:09.250717 5166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:09.265275 5171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:09.265399 5172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:09.265369 5174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:09.266708 5166 server_base.cc:1034] running on GCE node
I20260927 22:57:09.267423 5166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:09.268926 5166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:09.270560 5166 hybrid_clock.cc:648] HybridClock initialized: now 1790549829270468 us; error 83 us; skew 500 ppm
I20260927 22:57:09.271083 5166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:09 dist-test-slave-5pd3 krb5kdc[4926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549829, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:09.274713 5166 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:57:09.276477 5166 webserver.cc:460] Webserver started at http://127.4.74.130:35055/ using document root <none> and password file <none>
I20260927 22:57:09.277254 5166 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:09.277359 5166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:09.277621 5166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:09.280779 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "5ff5a41f75f34f5b82297b7d311708fa"
format_stamp: "Formatted at 2026-09-27 22:57:09 on dist-test-slave-5pd3"
server_key: "c7040d9ea908f52e52bba8d56b19c8f7"
server_key_iv: "aee821f0208fe62076b27bb88dce432f"
server_key_version: "encryptionkey@0"
I20260927 22:57:09.281634 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "5ff5a41f75f34f5b82297b7d311708fa"
format_stamp: "Formatted at 2026-09-27 22:57:09 on dist-test-slave-5pd3"
server_key: "c7040d9ea908f52e52bba8d56b19c8f7"
server_key_iv: "aee821f0208fe62076b27bb88dce432f"
server_key_version: "encryptionkey@0"
I20260927 22:57:09.287408 5166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 22:57:09.291927 5181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:09.293764 5166 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:57:09.293988 5166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "5ff5a41f75f34f5b82297b7d311708fa"
format_stamp: "Formatted at 2026-09-27 22:57:09 on dist-test-slave-5pd3"
server_key: "c7040d9ea908f52e52bba8d56b19c8f7"
server_key_iv: "aee821f0208fe62076b27bb88dce432f"
server_key_version: "encryptionkey@0"
I20260927 22:57:09.294224 5166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:09.312462 5166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:09.317749 5166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:09.318238 5166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:09.320098 5166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:09.321852 5166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:09.321941 5166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:09.322098 5166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:09.322163 5166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:09.361195 5166 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:39435
I20260927 22:57:09.361227 5293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:39435 every 8 connection(s)
I20260927 22:57:09.362551 5166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:57:09.363758 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 5166
I20260927 22:57:09.364015 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:57:09.364648 4394 external_mini_cluster.cc:1428] Setting key ed2e27b48322df04789182ff4133e2dd
Sep 27 22:57:09 dist-test-slave-5pd3 krb5kdc[4926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549829, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 22:57:09.385727 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:09.366325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:52485 (local address 127.4.74.190:38075)
0927 22:57:09.366910 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:09.366917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:09.368081 (+ 1164us) server_negotiation.cc:407] Connection header received
0927 22:57:09.370093 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:09.370103 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:09.370222 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:09.370414 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:09.372197 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:09.373016 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:09.374749 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:09.375195 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:09.375336 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:09.379073 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:09.379079 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:09.379082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:09.379119 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:09.382212 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:09.382996 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:09.383008 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:09.383011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:09.383135 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:09.383593 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:09.383600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:09.383603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:09.383882 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:09.384044 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:09.385097 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0927 22:57:09.385348 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":277,"threads_started":1}
I20260927 22:57:09.386346 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:09.366885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:38075 (local address 127.4.74.130:52485)
0927 22:57:09.367853 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:09.367924 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:09.369740 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:09.370718 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:09.370727 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:09.371311 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:09.371979 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:09.371996 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:09.373190 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:09.373201 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:09.374596 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:09.374608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:09.375427 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:09.375436 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:09.375741 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:09.376575 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:09.376598 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:09.378815 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:09.382377 (+ 3562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:09.382387 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:09.382406 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:09.382865 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:09.383248 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:09.383254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:09.383258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:09.383446 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:09.384012 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:09.384020 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:09.384403 (+ 383us) client_negotiation.cc:769] Sending connection context
0927 22:57:09.384681 (+ 278us) client_negotiation.cc:240] Negotiation successful
0927 22:57:09.385102 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":372,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
I20260927 22:57:09.388629 5294 heartbeater.cc:347] Connected to a master server at 127.4.74.190:38075
I20260927 22:57:09.389020 5294 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:09.389848 5294 heartbeater.cc:511] Master 127.4.74.190:38075 requested a full tablet report, sending...
I20260927 22:57:09.391897 4971 ts_manager.cc:194] Registered new tserver with Master: 5ff5a41f75f34f5b82297b7d311708fa (127.4.74.130:39435)
I20260927 22:57:09.392762 4971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:52485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:57:09.417608 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:38075
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:09.825482 5300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:09.825891 5300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:09.826076 5300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:09.835457 5300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:09.835736 5300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:09.835949 5300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:57:09.840456 5160 heartbeater.cc:503] Master 127.4.74.190:38075 was elected leader, sending a full tablet report...
I20260927 22:57:09.848740 5300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:38075
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:09.850605 5300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:09.852510 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:09.871654 5305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:09.871611 5308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:09.871886 5306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:09.873415 5300 server_base.cc:1034] running on GCE node
I20260927 22:57:09.874336 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:09.875644 5300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:09.876904 5300 hybrid_clock.cc:648] HybridClock initialized: now 1790549829876817 us; error 82 us; skew 500 ppm
I20260927 22:57:09.877328 5300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:09 dist-test-slave-5pd3 krb5kdc[4926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549829, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:09.882112 5300 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:57:09.884508 5300 webserver.cc:460] Webserver started at http://127.4.74.131:33883/ using document root <none> and password file <none>
I20260927 22:57:09.885468 5300 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:09.885619 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:09.886029 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:09.890019 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "0f1cbf184f364762911bfb2437aa59de"
format_stamp: "Formatted at 2026-09-27 22:57:09 on dist-test-slave-5pd3"
server_key: "321b0696e8888745af4b502e3a3d18ed"
server_key_iv: "4261f637054baae4bb412c631fd4d6e0"
server_key_version: "encryptionkey@0"
I20260927 22:57:09.890964 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "0f1cbf184f364762911bfb2437aa59de"
format_stamp: "Formatted at 2026-09-27 22:57:09 on dist-test-slave-5pd3"
server_key: "321b0696e8888745af4b502e3a3d18ed"
server_key_iv: "4261f637054baae4bb412c631fd4d6e0"
server_key_version: "encryptionkey@0"
I20260927 22:57:09.897615 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260927 22:57:09.902601 5315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:09.905333 5300 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260927 22:57:09.905728 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "0f1cbf184f364762911bfb2437aa59de"
format_stamp: "Formatted at 2026-09-27 22:57:09 on dist-test-slave-5pd3"
server_key: "321b0696e8888745af4b502e3a3d18ed"
server_key_iv: "4261f637054baae4bb412c631fd4d6e0"
server_key_version: "encryptionkey@0"
I20260927 22:57:09.906000 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:09.927206 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:09.933964 5300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:09.934317 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:09.936038 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:09.938289 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:09.938403 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:09.938513 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:09.938581 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:09.978688 5300 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:35709
I20260927 22:57:09.978749 5427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:35709 every 8 connection(s)
I20260927 22:57:09.980445 5300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:57:09.983958 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 5300
I20260927 22:57:09.984237 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:57:09.984776 4394 external_mini_cluster.cc:1428] Setting key 18312cbcc2a2ad6f85617a04101732c7
Sep 27 22:57:09 dist-test-slave-5pd3 krb5kdc[4926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549829, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:10.021624 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:09.983772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:50835 (local address 127.4.74.190:38075)
0927 22:57:09.984419 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:09.984427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:09.985365 (+ 938us) server_negotiation.cc:407] Connection header received
0927 22:57:09.987457 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:09.987465 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:09.987624 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:09.987766 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:09.989681 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:09.990546 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:09.992401 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:09.992958 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:09.993107 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:09.997663 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:09.997671 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:09.997675 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:09.997766 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:10.002954 (+ 5188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.004987 (+ 2033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.004994 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.004997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.006241 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.007171 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.007176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.007178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.018529 (+ 11351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:10.019007 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.020734 (+ 1727us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.021105 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":314,"threads_started":1}
I20260927 22:57:10.025242 5431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:09.984259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:38075 (local address 127.4.74.131:50835)
0927 22:57:09.985171 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:09.985217 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:09.987175 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:09.988050 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:09.988061 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:09.988859 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:09.989475 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:09.989492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:09.990737 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:09.990741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:09.992193 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:09.992207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:09.993213 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:09.993219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:09.993456 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:09.994291 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:09.994315 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:09.997131 (+ 2816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:10.003299 (+ 6168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:10.003317 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:10.003350 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:10.004253 (+ 903us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:10.006520 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:10.006525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:10.006529 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:10.006915 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:10.019049 (+ 12134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:10.019066 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:10.019787 (+ 721us) client_negotiation.cc:769] Sending connection context
0927 22:57:10.020432 (+ 645us) client_negotiation.cc:240] Negotiation successful
0927 22:57:10.021029 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":356,"threads_started":1}
I20260927 22:57:10.028756 5428 heartbeater.cc:347] Connected to a master server at 127.4.74.190:38075
I20260927 22:57:10.029232 5428 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:10.030619 5428 heartbeater.cc:511] Master 127.4.74.190:38075 requested a full tablet report, sending...
I20260927 22:57:10.033510 4971 ts_manager.cc:194] Registered new tserver with Master: 0f1cbf184f364762911bfb2437aa59de (127.4.74.131:35709)
I20260927 22:57:10.034529 4971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:50835
I20260927 22:57:10.036774 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 22:57:10.069455 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.047951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41788 (local address 127.4.74.190:38075)
0927 22:57:10.048290 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.048302 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.049274 (+ 972us) server_negotiation.cc:407] Connection header received
0927 22:57:10.049386 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.049391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.049489 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.049633 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:10.052280 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.053540 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.055498 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.056118 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.056284 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.059215 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:10.059223 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:10.059227 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:10.059285 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:10.063778 (+ 4493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.064953 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.064959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.064962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.065158 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.065869 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.065874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.065877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.066947 (+ 1070us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:10.067270 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.068318 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.068981 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260927 22:57:10.081073 4971 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:57:10.089600 4971 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:10.139837 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.109762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:41987 (local address 127.0.0.1:46622)
0927 22:57:10.128179 (+ 18417us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:10.128208 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:10.128437 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:10.130286 (+ 1849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:10.130295 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:10.130455 (+ 160us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:10.131039 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:10.131056 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.133258 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.133265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:10.135954 (+ 2689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:10.135974 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.138644 (+ 2670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.138653 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:10.138915 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.138976 (+ 61us) client_negotiation.cc:769] Sending connection context
0927 22:57:10.139150 (+ 174us) client_negotiation.cc:240] Negotiation successful
0927 22:57:10.139410 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18227,"thread_start_us":13819,"threads_started":1}
I20260927 22:57:10.140787 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.110234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46622 (local address 127.4.74.129:41987)
0927 22:57:10.121943 (+ 11709us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.121953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.128627 (+ 6674us) server_negotiation.cc:407] Connection header received
0927 22:57:10.128759 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.128768 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.129121 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.129698 (+ 577us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:10.131314 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.133048 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.136194 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.137839 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.137980 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.138375 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.140230 (+ 1855us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.140434 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11456,"thread_start_us":11235,"threads_started":1}
I20260927 22:57:10.145043 5095 tablet_service.cc:1467] Processing CreateTablet for tablet f33ee527c3b4424e9d1828f2b20b38aa (DEFAULT_TABLE table=test-table [id=480bb40f22ad430b933c4946a76486a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:10.146908 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f33ee527c3b4424e9d1828f2b20b38aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:10.148633 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.109513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:35709 (local address 127.0.0.1:49096)
0927 22:57:10.121259 (+ 11746us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:10.121325 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:10.121580 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:10.132295 (+ 10715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:10.132304 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:10.132356 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:10.133038 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:10.133057 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.138676 (+ 5619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.138687 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:10.144666 (+ 5979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:10.144782 (+ 116us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.147796 (+ 3014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.147803 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:10.148033 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.148073 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 22:57:10.148137 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 22:57:10.148266 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11394,"thread_start_us":18288,"threads_started":1}
I20260927 22:57:10.149765 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.109859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.4.74.131:35709)
0927 22:57:10.127689 (+ 17830us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.127706 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.127768 (+ 62us) server_negotiation.cc:407] Connection header received
0927 22:57:10.127947 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.127957 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.128577 (+ 620us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.133371 (+ 4794us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:10.133844 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.138142 (+ 4298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.145165 (+ 7023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.147039 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.148941 (+ 1902us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.149242 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.149321 (+ 79us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.149480 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17480,"thread_start_us":17210,"threads_started":1}
I20260927 22:57:10.152622 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.135075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:39435 (local address 127.0.0.1:56774)
0927 22:57:10.142365 (+ 7290us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:10.142390 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:10.142547 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:10.145279 (+ 2732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:10.145284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:10.145327 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:10.145530 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:10.145540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.147693 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.147700 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:10.149634 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:10.149656 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.151717 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.151724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:10.151937 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.151982 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 22:57:10.152057 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 22:57:10.152243 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7161,"thread_start_us":4940,"threads_started":1}
I20260927 22:57:10.153774 5445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.129347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56774 (local address 127.4.74.130:39435)
0927 22:57:10.144169 (+ 14822us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.144183 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.144224 (+ 41us) server_negotiation.cc:407] Connection header received
0927 22:57:10.144362 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.144372 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.144952 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.145236 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:10.145789 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.147431 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.149894 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.151450 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.152933 (+ 1483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.153180 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.153263 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.153441 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14569,"thread_start_us":16088,"threads_started":1}
I20260927 22:57:10.154597 5363 tablet_service.cc:1467] Processing CreateTablet for tablet f33ee527c3b4424e9d1828f2b20b38aa (DEFAULT_TABLE table=test-table [id=480bb40f22ad430b933c4946a76486a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:10.157048 5363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f33ee527c3b4424e9d1828f2b20b38aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:10.160053 5229 tablet_service.cc:1467] Processing CreateTablet for tablet f33ee527c3b4424e9d1828f2b20b38aa (DEFAULT_TABLE table=test-table [id=480bb40f22ad430b933c4946a76486a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:10.161213 5447 tablet_bootstrap.cc:492] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d: Bootstrap starting.
I20260927 22:57:10.161798 5229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f33ee527c3b4424e9d1828f2b20b38aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:10.165418 5447 tablet_bootstrap.cc:654] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:10.167366 5447 log.cc:824] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:10.170848 5447 tablet_bootstrap.cc:492] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d: No bootstrap required, opened a new log
I20260927 22:57:10.171206 5447 ts_tablet_manager.cc:1397] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260927 22:57:10.179505 5447 raft_consensus.cc:348] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1cbf184f364762911bfb2437aa59de" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35709 } } peers { permanent_uuid: "5ff5a41f75f34f5b82297b7d311708fa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39435 } } peers { permanent_uuid: "954b57b2f9324e0ab300279ae6fcc33d" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41987 } }
I20260927 22:57:10.180289 5447 raft_consensus.cc:374] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:10.180434 5447 raft_consensus.cc:729] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 954b57b2f9324e0ab300279ae6fcc33d, State: Initialized, Role: FOLLOWER
I20260927 22:57:10.181397 5450 tablet_bootstrap.cc:492] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de: Bootstrap starting.
I20260927 22:57:10.181447 5447 consensus_queue.cc:260] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1cbf184f364762911bfb2437aa59de" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35709 } } peers { permanent_uuid: "5ff5a41f75f34f5b82297b7d311708fa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39435 } } peers { permanent_uuid: "954b57b2f9324e0ab300279ae6fcc33d" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41987 } }
I20260927 22:57:10.182229 5449 tablet_bootstrap.cc:492] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa: Bootstrap starting.
I20260927 22:57:10.186419 5450 tablet_bootstrap.cc:654] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:10.187922 5450 log.cc:824] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:10.189328 5449 tablet_bootstrap.cc:654] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:10.189791 5447 ts_tablet_manager.cc:1428] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d: Time spent starting tablet: real 0.018s user 0.004s sys 0.015s
I20260927 22:57:10.191128 5449 log.cc:824] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:10.194258 5450 tablet_bootstrap.cc:492] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de: No bootstrap required, opened a new log
I20260927 22:57:10.194608 5450 ts_tablet_manager.cc:1397] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260927 22:57:10.197494 5449 tablet_bootstrap.cc:492] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa: No bootstrap required, opened a new log
I20260927 22:57:10.197997 5449 ts_tablet_manager.cc:1397] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.003s
I20260927 22:57:10.206768 5450 raft_consensus.cc:348] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1cbf184f364762911bfb2437aa59de" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35709 } } peers { permanent_uuid: "5ff5a41f75f34f5b82297b7d311708fa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39435 } } peers { permanent_uuid: "954b57b2f9324e0ab300279ae6fcc33d" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41987 } }
I20260927 22:57:10.207384 5450 raft_consensus.cc:374] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:10.207626 5450 raft_consensus.cc:729] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f1cbf184f364762911bfb2437aa59de, State: Initialized, Role: FOLLOWER
I20260927 22:57:10.208557 5450 consensus_queue.cc:260] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1cbf184f364762911bfb2437aa59de" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35709 } } peers { permanent_uuid: "5ff5a41f75f34f5b82297b7d311708fa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39435 } } peers { permanent_uuid: "954b57b2f9324e0ab300279ae6fcc33d" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41987 } }
I20260927 22:57:10.209718 5449 raft_consensus.cc:348] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1cbf184f364762911bfb2437aa59de" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35709 } } peers { permanent_uuid: "5ff5a41f75f34f5b82297b7d311708fa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39435 } } peers { permanent_uuid: "954b57b2f9324e0ab300279ae6fcc33d" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41987 } }
I20260927 22:57:10.210278 5449 raft_consensus.cc:374] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:10.210393 5449 raft_consensus.cc:729] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff5a41f75f34f5b82297b7d311708fa, State: Initialized, Role: FOLLOWER
I20260927 22:57:10.210490 5428 heartbeater.cc:503] Master 127.4.74.190:38075 was elected leader, sending a full tablet report...
I20260927 22:57:10.210781 5450 ts_tablet_manager.cc:1428] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de: Time spent starting tablet: real 0.016s user 0.015s sys 0.004s
I20260927 22:57:10.211211 5449 consensus_queue.cc:260] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1cbf184f364762911bfb2437aa59de" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35709 } } peers { permanent_uuid: "5ff5a41f75f34f5b82297b7d311708fa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39435 } } peers { permanent_uuid: "954b57b2f9324e0ab300279ae6fcc33d" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41987 } }
I20260927 22:57:10.212801 5294 heartbeater.cc:503] Master 127.4.74.190:38075 was elected leader, sending a full tablet report...
I20260927 22:57:10.213354 5449 ts_tablet_manager.cc:1428] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260927 22:57:10.232740 5429 tablet.cc:2369] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:57:10.249512 5455 raft_consensus.cc:482] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:10.249898 5455 raft_consensus.cc:504] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1cbf184f364762911bfb2437aa59de" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35709 } } peers { permanent_uuid: "5ff5a41f75f34f5b82297b7d311708fa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39435 } } peers { permanent_uuid: "954b57b2f9324e0ab300279ae6fcc33d" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41987 } }
I20260927 22:57:10.251813 5455 leader_election.cc:288] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f1cbf184f364762911bfb2437aa59de (127.4.74.131:35709), 954b57b2f9324e0ab300279ae6fcc33d (127.4.74.129:41987)
I20260927 22:57:10.259716 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.252569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:37893 (local address 127.4.74.131:35709)
0927 22:57:10.252882 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.252893 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.253736 (+ 843us) server_negotiation.cc:407] Connection header received
0927 22:57:10.253979 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.253986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.254052 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.254162 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:10.254965 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.255800 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.257917 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.258711 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.258917 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.258997 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.259271 (+ 274us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.259400 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260927 22:57:10.259902 5457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.252587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:35709 (local address 127.4.74.130:37893)
0927 22:57:10.253598 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:10.253618 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:10.253823 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:10.254182 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:10.254186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:10.254214 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:10.254800 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:10.254812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.255938 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.255946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:10.257759 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:10.257773 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.258933 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.258939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:10.259104 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.259129 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 22:57:10.259246 (+ 117us) client_negotiation.cc:240] Negotiation successful
0927 22:57:10.259398 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":88,"thread_start_us":595,"threads_started":1}
I20260927 22:57:10.260743 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.252327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:38905 (local address 127.4.74.129:41987)
0927 22:57:10.252563 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.252571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.253665 (+ 1094us) server_negotiation.cc:407] Connection header received
0927 22:57:10.253928 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.253932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.254007 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.254116 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:10.254897 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.255806 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.258151 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.259061 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.259284 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.259389 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.260241 (+ 852us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.260432 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 22:57:10.261257 5456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.252437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:41987 (local address 127.4.74.130:38905)
0927 22:57:10.253476 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:10.253511 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:10.253737 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:10.254142 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:10.254146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:10.254174 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:10.254719 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:10.254730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.255950 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.255956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:10.257940 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:10.257958 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.259327 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.259334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:10.259518 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.259633 (+ 115us) client_negotiation.cc:769] Sending connection context
0927 22:57:10.260640 (+ 1007us) client_negotiation.cc:240] Negotiation successful
0927 22:57:10.260816 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":3,"thread_start_us":588,"threads_started":1}
I20260927 22:57:10.261593 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33ee527c3b4424e9d1828f2b20b38aa" candidate_uuid: "5ff5a41f75f34f5b82297b7d311708fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1cbf184f364762911bfb2437aa59de" is_pre_election: true
I20260927 22:57:10.262280 5383 raft_consensus.cc:2454] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff5a41f75f34f5b82297b7d311708fa in term 0.
I20260927 22:57:10.262487 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33ee527c3b4424e9d1828f2b20b38aa" candidate_uuid: "5ff5a41f75f34f5b82297b7d311708fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954b57b2f9324e0ab300279ae6fcc33d" is_pre_election: true
I20260927 22:57:10.263092 5115 raft_consensus.cc:2454] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ff5a41f75f34f5b82297b7d311708fa in term 0.
I20260927 22:57:10.263128 5182 leader_election.cc:302] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f1cbf184f364762911bfb2437aa59de, 5ff5a41f75f34f5b82297b7d311708fa; no voters:
I20260927 22:57:10.263595 5455 raft_consensus.cc:2781] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:10.263804 5455 raft_consensus.cc:482] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:10.263926 5455 raft_consensus.cc:3037] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:10.266227 5455 raft_consensus.cc:504] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1cbf184f364762911bfb2437aa59de" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35709 } } peers { permanent_uuid: "5ff5a41f75f34f5b82297b7d311708fa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39435 } } peers { permanent_uuid: "954b57b2f9324e0ab300279ae6fcc33d" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41987 } }
I20260927 22:57:10.266876 5455 leader_election.cc:288] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [CANDIDATE]: Term 1 election: Requested vote from peers 0f1cbf184f364762911bfb2437aa59de (127.4.74.131:35709), 954b57b2f9324e0ab300279ae6fcc33d (127.4.74.129:41987)
I20260927 22:57:10.267660 5383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33ee527c3b4424e9d1828f2b20b38aa" candidate_uuid: "5ff5a41f75f34f5b82297b7d311708fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f1cbf184f364762911bfb2437aa59de"
I20260927 22:57:10.267678 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33ee527c3b4424e9d1828f2b20b38aa" candidate_uuid: "5ff5a41f75f34f5b82297b7d311708fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954b57b2f9324e0ab300279ae6fcc33d"
I20260927 22:57:10.267971 5383 raft_consensus.cc:3037] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:10.267971 5115 raft_consensus.cc:3037] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:10.271842 5383 raft_consensus.cc:2454] T f33ee527c3b4424e9d1828f2b20b38aa P 0f1cbf184f364762911bfb2437aa59de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff5a41f75f34f5b82297b7d311708fa in term 1.
I20260927 22:57:10.271842 5115 raft_consensus.cc:2454] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ff5a41f75f34f5b82297b7d311708fa in term 1.
I20260927 22:57:10.272677 5185 leader_election.cc:302] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ff5a41f75f34f5b82297b7d311708fa, 954b57b2f9324e0ab300279ae6fcc33d; no voters:
I20260927 22:57:10.273283 5455 raft_consensus.cc:2781] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:10.273888 5455 raft_consensus.cc:686] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [term 1 LEADER]: Becoming Leader. State: Replica: 5ff5a41f75f34f5b82297b7d311708fa, State: Running, Role: LEADER
I20260927 22:57:10.274544 5455 consensus_queue.cc:237] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1cbf184f364762911bfb2437aa59de" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35709 } } peers { permanent_uuid: "5ff5a41f75f34f5b82297b7d311708fa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39435 } } peers { permanent_uuid: "954b57b2f9324e0ab300279ae6fcc33d" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41987 } }
I20260927 22:57:10.280159 4970 catalog_manager.cc:5495] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ff5a41f75f34f5b82297b7d311708fa (127.4.74.130). New cstate: current_term: 1 leader_uuid: "5ff5a41f75f34f5b82297b7d311708fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f1cbf184f364762911bfb2437aa59de" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ff5a41f75f34f5b82297b7d311708fa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "954b57b2f9324e0ab300279ae6fcc33d" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41987 } health_report { overall_health: UNKNOWN } } }
W20260927 22:57:10.294399 5161 tablet.cc:2369] T f33ee527c3b4424e9d1828f2b20b38aa P 954b57b2f9324e0ab300279ae6fcc33d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 22:57:10 dist-test-slave-5pd3 krb5kdc[4926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:10.311648 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.294502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.4.74.129:41987)
0927 22:57:10.294774 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.294785 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.295384 (+ 599us) server_negotiation.cc:407] Connection header received
0927 22:57:10.295615 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.295622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.295740 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.295921 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:10.297424 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.298419 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.299773 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.300227 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.300335 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.304899 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:10.304919 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:10.304938 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:10.304987 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:10.308301 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.309187 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.309195 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.309198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.309350 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.309738 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.309745 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.309749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.310171 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:10.310542 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.310854 (+ 312us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.311179 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
Sep 27 22:57:10 dist-test-slave-5pd3 krb5kdc[4926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 22:57:10 dist-test-slave-5pd3 krb5kdc[4926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
W20260927 22:57:10.365361 5295 tablet.cc:2369] T f33ee527c3b4424e9d1828f2b20b38aa P 5ff5a41f75f34f5b82297b7d311708fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:57:10.368602 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.350809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46638 (local address 127.4.74.129:41987)
0927 22:57:10.351104 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.351110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.351312 (+ 202us) server_negotiation.cc:407] Connection header received
0927 22:57:10.351584 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.351654 (+ 70us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.351740 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.351854 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:10.353323 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.354112 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.355469 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.355973 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.356092 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.360644 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:10.360658 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:10.360664 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:10.360716 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:10.364866 (+ 4150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.366068 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.366074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.366077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.366268 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.366782 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.366787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.366792 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.367169 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:10.367448 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.367877 (+ 429us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.368190 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
Sep 27 22:57:10 dist-test-slave-5pd3 krb5kdc[4926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 22:57:10 dist-test-slave-5pd3 krb5kdc[4926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:10.423723 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.406456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.4.74.129:41987)
0927 22:57:10.406811 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.406821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.407064 (+ 243us) server_negotiation.cc:407] Connection header received
0927 22:57:10.407294 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.407299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.407396 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.407510 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:10.409368 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.410223 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.411944 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.412430 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.412579 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.417414 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:10.417420 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:10.417424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:10.417468 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:10.420537 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.421535 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.421539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.421542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.421725 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.422121 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.422124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.422126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.422452 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:10.422678 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.423087 (+ 409us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.423297 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
W20260927 22:57:10.425567 5075 tablet_service.cc:2908] Rejecting scan request for tablet f33ee527c3b4424e9d1828f2b20b38aa: Uninitialized: clean time has not yet been initialized
I20260927 22:57:10.429840 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 5032
I20260927 22:57:10.446247 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.431192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.4.74.129:41987)
0927 22:57:10.431719 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.431727 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.432056 (+ 329us) server_negotiation.cc:407] Connection header received
0927 22:57:10.432199 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.432203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.432308 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.432557 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:10.434075 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.434793 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.436389 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.436861 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.436998 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.439829 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:10.439834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:10.439838 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:10.439876 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:10.443140 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.444243 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.444247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.444250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.444403 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.444826 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.444830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.444832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.445147 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:10.445376 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.445751 (+ 375us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.445944 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260927 22:57:10.447151 5155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:10.541790 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 5032
I20260927 22:57:10.559650 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 5166
Sep 27 22:57:10 dist-test-slave-5pd3 krb5kdc[4926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:57:10.576871 5445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.560616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.4.74.130:39435)
0927 22:57:10.560825 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.560831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.560854 (+ 23us) server_negotiation.cc:407] Connection header received
0927 22:57:10.560934 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.560938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.561011 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.561135 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:10.562451 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.563242 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.564762 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.565208 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.565323 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.569627 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:10.569646 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:10.569664 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:10.569709 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:10.573548 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.574588 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.574597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.574600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.574813 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.575198 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.575202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.575204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.575744 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:10.576028 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.576340 (+ 312us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.576588 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 22:57:10.577761 5289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:10.681063 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 5166
I20260927 22:57:10.698568 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 5300
Sep 27 22:57:10 dist-test-slave-5pd3 krb5kdc[4926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:57:10.716226 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:10.699611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.4.74.131:35709)
0927 22:57:10.699901 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:10.699907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:10.699933 (+ 26us) server_negotiation.cc:407] Connection header received
0927 22:57:10.700022 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:10.700027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:10.700111 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:10.700233 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:10.701595 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.702520 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.704078 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:10.704584 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:10.704721 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:10.709240 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:10.709260 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:10.709281 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:10.709337 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:10.713106 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.713965 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.713974 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.713977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.714137 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:10.714530 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:10.714534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:10.714537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:10.714975 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:10.715236 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:10.715479 (+ 243us) server_negotiation.cc:299] Negotiation successful
0927 22:57:10.715772 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260927 22:57:10.717000 5423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:10.814679 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 5300
I20260927 22:57:10.831969 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 4939
I20260927 22:57:10.833487 5001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:10.946164 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 4939
2026-09-27T22:57:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4621 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:11 dist-test-slave-5pd3 krb5kdc[5477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:11 dist-test-slave-5pd3 krb5kdc[5477](info): set up 2 sockets
Sep 27 22:57:11 dist-test-slave-5pd3 krb5kdc[5477](info): commencing operation
krb5kdc: starting...
W20260927 22:57:12.055969 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:12 dist-test-slave-5pd3 krb5kdc[5477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:57:12.267900 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:39797
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:39461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:39797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:12.619380 5490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:12.619786 5490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:12.619959 5490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:12.627403 5490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:12.627527 5490 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 22:57:12.627646 5490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:12.627820 5490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:12.627907 5490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:12.628006 5490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:12.638254 5490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:39461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:39797
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:39797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:12.639806 5490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:12.641196 5490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:12.654654 5496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:12.654801 5498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:12.655053 5495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:12.655751 5490 server_base.cc:1034] running on GCE node
I20260927 22:57:12.656632 5490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:12.658258 5490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:12.659531 5490 hybrid_clock.cc:648] HybridClock initialized: now 1790549832659461 us; error 56 us; skew 500 ppm
I20260927 22:57:12.659956 5490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:12 dist-test-slave-5pd3 krb5kdc[5477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549832, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:12.663298 5490 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:57:12.665125 5490 webserver.cc:460] Webserver started at http://127.4.74.190:46115/ using document root <none> and password file <none>
I20260927 22:57:12.665885 5490 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:12.666026 5490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:12.666361 5490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:12.669787 5490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "a601c50393e04f46bef9a0364f2792ec"
format_stamp: "Formatted at 2026-09-27 22:57:12 on dist-test-slave-5pd3"
server_key: "50fcd6fccd4cb5f85fdec9a5ca66e08e"
server_key_iv: "247ce60ce9e865cf9c80883bdd773922"
server_key_version: "encryptionkey@0"
I20260927 22:57:12.670637 5490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "a601c50393e04f46bef9a0364f2792ec"
format_stamp: "Formatted at 2026-09-27 22:57:12 on dist-test-slave-5pd3"
server_key: "50fcd6fccd4cb5f85fdec9a5ca66e08e"
server_key_iv: "247ce60ce9e865cf9c80883bdd773922"
server_key_version: "encryptionkey@0"
I20260927 22:57:12.676429 5490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260927 22:57:12.680775 5505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:12.682945 5490 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:57:12.683202 5490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "a601c50393e04f46bef9a0364f2792ec"
format_stamp: "Formatted at 2026-09-27 22:57:12 on dist-test-slave-5pd3"
server_key: "50fcd6fccd4cb5f85fdec9a5ca66e08e"
server_key_iv: "247ce60ce9e865cf9c80883bdd773922"
server_key_version: "encryptionkey@0"
I20260927 22:57:12.683459 5490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:12.696537 5490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:12.701551 5490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:12.701862 5490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:12.726320 5490 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:39797
I20260927 22:57:12.726372 5566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:39797 every 8 connection(s)
I20260927 22:57:12.727833 5490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:12.731083 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 5490
I20260927 22:57:12.731364 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:12.732304 4394 external_mini_cluster.cc:1428] Setting key 7ad6fcd6e7669fd275f4e38fe04ccaa4
I20260927 22:57:12.733696 5567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 22:57:12 dist-test-slave-5pd3 krb5kdc[5477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:12.751175 5567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec: Bootstrap starting.
I20260927 22:57:12.756839 5567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:12.758666 5567 log.cc:824] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:12.761292 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:12.734898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.4.74.190:39797)
0927 22:57:12.735835 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:12.735848 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:12.735897 (+ 49us) server_negotiation.cc:407] Connection header received
0927 22:57:12.737543 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:12.737566 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:12.738026 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:12.738403 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:12.740553 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:12.742017 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:12.744004 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:12.744785 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:12.744948 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:12.750630 (+ 5682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:12.750643 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:12.750670 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:12.750743 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:12.755291 (+ 4548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:12.756277 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:12.756287 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:12.756300 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:12.756776 (+ 476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:12.757335 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:12.757342 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:12.757346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:12.758294 (+ 948us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:12.758672 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:12.759139 (+ 467us) server_negotiation.cc:299] Negotiation successful
0927 22:57:12.759813 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":443,"threads_started":1}
I20260927 22:57:12.762792 5567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec: No bootstrap required, opened a new log
I20260927 22:57:12.772430 5567 raft_consensus.cc:348] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a601c50393e04f46bef9a0364f2792ec" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39797 } }
I20260927 22:57:12.773059 5567 raft_consensus.cc:374] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:12.773185 5567 raft_consensus.cc:729] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a601c50393e04f46bef9a0364f2792ec, State: Initialized, Role: FOLLOWER
I20260927 22:57:12.773944 5567 consensus_queue.cc:260] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a601c50393e04f46bef9a0364f2792ec" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39797 } }
I20260927 22:57:12.774250 5567 raft_consensus.cc:388] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:12.774407 5567 raft_consensus.cc:482] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:12.774602 5567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:12.777210 5567 raft_consensus.cc:504] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a601c50393e04f46bef9a0364f2792ec" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39797 } }
I20260927 22:57:12.778044 5567 leader_election.cc:302] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a601c50393e04f46bef9a0364f2792ec; no voters:
I20260927 22:57:12.778771 5567 leader_election.cc:288] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:12.778967 5572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:12.780068 5572 raft_consensus.cc:686] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [term 1 LEADER]: Becoming Leader. State: Replica: a601c50393e04f46bef9a0364f2792ec, State: Running, Role: LEADER
I20260927 22:57:12.780694 5572 consensus_queue.cc:237] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a601c50393e04f46bef9a0364f2792ec" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39797 } }
I20260927 22:57:12.781577 5567 sys_catalog.cc:564] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:12.784123 5574 sys_catalog.cc:455] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader a601c50393e04f46bef9a0364f2792ec. Latest consensus state: current_term: 1 leader_uuid: "a601c50393e04f46bef9a0364f2792ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a601c50393e04f46bef9a0364f2792ec" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39797 } } }
I20260927 22:57:12.784567 5574 sys_catalog.cc:458] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:12.785136 5573 sys_catalog.cc:455] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a601c50393e04f46bef9a0364f2792ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a601c50393e04f46bef9a0364f2792ec" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39797 } } }
I20260927 22:57:12.785375 5573 sys_catalog.cc:458] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:12.788906 5578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:12.796166 5578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:12.807395 5578 catalog_manager.cc:1347] Generated new cluster ID: f88d83dc8137430b8f616b1f21eb4eb4
I20260927 22:57:12.807605 5578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:12.841917 5578 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:12.843713 5578 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:12.856369 5578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec: Generated new TSK 0
I20260927 22:57:12.857571 5578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:57:12.920857 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39797
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:39461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:13.336141 5593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:13.336520 5593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:13.336711 5593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:13.345782 5593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:13.345999 5593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:13.346192 5593 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 22:57:13.346334 5593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:57:13.358692 5593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:39461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.74.190:39797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:13.360714 5593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:13.362458 5593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:13.378608 5601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:13.378572 5599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:13.378908 5598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:13.380198 5593 server_base.cc:1034] running on GCE node
I20260927 22:57:13.381026 5593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:13.382390 5593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:13.383783 5593 hybrid_clock.cc:648] HybridClock initialized: now 1790549833383710 us; error 45 us; skew 500 ppm
I20260927 22:57:13.384189 5593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:13 dist-test-slave-5pd3 krb5kdc[5477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549833, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:13.394570 5593 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:57:13.396829 5593 webserver.cc:460] Webserver started at http://127.4.74.129:35783/ using document root <none> and password file <none>
I20260927 22:57:13.397853 5593 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:13.397974 5593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:13.398403 5593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:13.402252 5593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "e00ea55ce1714046a8a23fd8cd77a8f7"
format_stamp: "Formatted at 2026-09-27 22:57:13 on dist-test-slave-5pd3"
server_key: "a1cee2fb44bf0ed5372115cd08802251"
server_key_iv: "178b50e292a8051f9c7eb4fa701217a7"
server_key_version: "encryptionkey@0"
I20260927 22:57:13.403301 5593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "e00ea55ce1714046a8a23fd8cd77a8f7"
format_stamp: "Formatted at 2026-09-27 22:57:13 on dist-test-slave-5pd3"
server_key: "a1cee2fb44bf0ed5372115cd08802251"
server_key_iv: "178b50e292a8051f9c7eb4fa701217a7"
server_key_version: "encryptionkey@0"
I20260927 22:57:13.410254 5593 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 22:57:13.415323 5608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:13.417354 5593 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:57:13.417639 5593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "e00ea55ce1714046a8a23fd8cd77a8f7"
format_stamp: "Formatted at 2026-09-27 22:57:13 on dist-test-slave-5pd3"
server_key: "a1cee2fb44bf0ed5372115cd08802251"
server_key_iv: "178b50e292a8051f9c7eb4fa701217a7"
server_key_version: "encryptionkey@0"
I20260927 22:57:13.417956 5593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:13.436362 5593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:13.442147 5593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:13.442514 5593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:13.446316 5593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:13.446457 5593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:13.446646 5593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:13.446735 5593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 22:57:13 dist-test-slave-5pd3 krb5kdc[5477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549833, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:13.475378 5617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:13.446943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:38013 (local address 127.4.74.190:39797)
0927 22:57:13.447881 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:13.447886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:13.449256 (+ 1370us) server_negotiation.cc:407] Connection header received
0927 22:57:13.451763 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:13.451769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:13.451858 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:13.451998 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:13.454717 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:13.455770 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:13.458927 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:13.459357 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:13.459506 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:13.466397 (+ 6891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:13.466403 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:13.466407 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:13.466451 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:13.470637 (+ 4186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:13.472143 (+ 1506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:13.472150 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:13.472154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:13.472380 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:13.473278 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:13.473286 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:13.473291 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:13.473710 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:13.473883 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:13.474736 (+ 853us) server_negotiation.cc:299] Negotiation successful
0927 22:57:13.474987 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":729,"threads_started":1}
I20260927 22:57:13.476679 5618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:13.447888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39797 (local address 127.4.74.129:38013)
0927 22:57:13.449033 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:13.449086 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:13.451342 (+ 2256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:13.452312 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:13.452327 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:13.453363 (+ 1036us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:13.454495 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:13.454516 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:13.455947 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:13.455962 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:13.458740 (+ 2778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:13.458758 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:13.459679 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:13.459688 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:13.459935 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:13.461029 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:13.461072 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:13.465982 (+ 4910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:13.470987 (+ 5005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:13.471003 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:13.471032 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:13.471944 (+ 912us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:13.472578 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:13.472588 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:13.472595 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:13.473072 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:13.473880 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:13.473890 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:13.474321 (+ 431us) client_negotiation.cc:769] Sending connection context
0927 22:57:13.474630 (+ 309us) client_negotiation.cc:240] Negotiation successful
0927 22:57:13.475205 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":490,"threads_started":1}
I20260927 22:57:13.503432 5617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:13.484594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:38871 (local address 127.4.74.190:39797)
0927 22:57:13.484860 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:13.484868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:13.485579 (+ 711us) server_negotiation.cc:407] Connection header received
0927 22:57:13.486015 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:13.486024 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:13.486130 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:13.486251 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:13.488795 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:13.489614 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:13.492372 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:13.492824 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:13.492939 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:13.496333 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:13.496343 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:13.496349 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:13.496399 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:13.499498 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:13.500878 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:13.500884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:13.500888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:13.501070 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:13.501692 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:13.501697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:13.501701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:13.502004 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:13.502192 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:13.502945 (+ 753us) server_negotiation.cc:299] Negotiation successful
0927 22:57:13.503185 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260927 22:57:13.504129 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:13.484517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39797 (local address 127.4.74.129:38871)
0927 22:57:13.485372 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:13.485410 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:13.485759 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:13.486425 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:13.486429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:13.487708 (+ 1279us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:13.488567 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:13.488595 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:13.489822 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:13.489831 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:13.492159 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:13.492181 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:13.493070 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:13.493078 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:13.493325 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:13.494373 (+ 1048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:13.494407 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:13.496034 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:13.499710 (+ 3676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:13.499716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:13.499721 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:13.500669 (+ 948us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:13.501233 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:13.501239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:13.501246 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:13.501507 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:13.502182 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:13.502187 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:13.502564 (+ 377us) client_negotiation.cc:769] Sending connection context
0927 22:57:13.502789 (+ 225us) client_negotiation.cc:240] Negotiation successful
0927 22:57:13.503342 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":399,"threads_started":1}
I20260927 22:57:13.512778 5593 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:37123
I20260927 22:57:13.512816 5728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:37123 every 8 connection(s)
I20260927 22:57:13.514775 5593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:57:13.516667 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 5593
I20260927 22:57:13.517040 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:57:13.518045 4394 external_mini_cluster.cc:1428] Setting key 8be4c8d16e9524ff1d0b3fe722aa087b
I20260927 22:57:13.518111 5729 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39797
I20260927 22:57:13.518419 5729 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:13.519332 5729 heartbeater.cc:511] Master 127.4.74.190:39797 requested a full tablet report, sending...
I20260927 22:57:13.521958 5522 ts_manager.cc:194] Registered new tserver with Master: e00ea55ce1714046a8a23fd8cd77a8f7 (127.4.74.129:37123)
I20260927 22:57:13.524365 5522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:38013
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:57:13.564378 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39797
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:39461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:13.922485 5733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:13.922849 5733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:13.923031 5733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:13.930822 5733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:13.930951 5733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:13.931088 5733 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 22:57:13.931245 5733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:57:13.941411 5733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:39461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.74.190:39797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:13.942906 5733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:13.944512 5733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:13.960022 5739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:13.960073 5741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:13.960137 5738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:13.965787 5733 server_base.cc:1034] running on GCE node
I20260927 22:57:13.966817 5733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:13.968302 5733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:13.969928 5733 hybrid_clock.cc:648] HybridClock initialized: now 1790549833969790 us; error 117 us; skew 500 ppm
I20260927 22:57:13.970573 5733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:13 dist-test-slave-5pd3 krb5kdc[5477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549833, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:13.975288 5733 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:57:13.977615 5733 webserver.cc:460] Webserver started at http://127.4.74.130:38165/ using document root <none> and password file <none>
I20260927 22:57:13.978660 5733 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:13.978825 5733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:13.979336 5733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:13.983839 5733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e"
format_stamp: "Formatted at 2026-09-27 22:57:13 on dist-test-slave-5pd3"
server_key: "4c38b30edc5fef00887f08b07d4e730b"
server_key_iv: "335675a588dc962739bb47097090eb42"
server_key_version: "encryptionkey@0"
I20260927 22:57:13.984962 5733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e"
format_stamp: "Formatted at 2026-09-27 22:57:13 on dist-test-slave-5pd3"
server_key: "4c38b30edc5fef00887f08b07d4e730b"
server_key_iv: "335675a588dc962739bb47097090eb42"
server_key_version: "encryptionkey@0"
I20260927 22:57:13.993382 5733 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.005s
I20260927 22:57:14.000058 5748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:14.002243 5733 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260927 22:57:14.002556 5733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e"
format_stamp: "Formatted at 2026-09-27 22:57:13 on dist-test-slave-5pd3"
server_key: "4c38b30edc5fef00887f08b07d4e730b"
server_key_iv: "335675a588dc962739bb47097090eb42"
server_key_version: "encryptionkey@0"
I20260927 22:57:14.002905 5733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:14.031121 5733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:14.036449 5733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:14.036809 5733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:14.040472 5733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:14.040619 5733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:14.040843 5733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:14.040915 5733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 22:57:14 dist-test-slave-5pd3 krb5kdc[5477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549833, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:14.066636 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.040917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:48271 (local address 127.4.74.190:39797)
0927 22:57:14.041479 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:14.041486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:14.042944 (+ 1458us) server_negotiation.cc:407] Connection header received
0927 22:57:14.045480 (+ 2536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:14.045490 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:14.045618 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:14.045817 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:14.048496 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.049616 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.052438 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.053028 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.053167 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.060044 (+ 6877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:14.060050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:14.060053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:14.060096 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:14.062725 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:14.063515 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:14.063520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:14.063526 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:14.063705 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:14.064367 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:14.064373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:14.064376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:14.064719 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:14.064875 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:14.066010 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0927 22:57:14.066238 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":280,"threads_started":1}
I20260927 22:57:14.068011 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.041646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39797 (local address 127.4.74.130:48271)
0927 22:57:14.042702 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:14.042769 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:14.045020 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:14.046256 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:14.046269 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:14.047117 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:14.048256 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.048291 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.049787 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.049794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.052285 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.052301 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.053347 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.053355 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.053593 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.054852 (+ 1259us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:14.054911 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:14.059715 (+ 4804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:14.062882 (+ 3167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:14.062892 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:14.062910 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:14.063384 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:14.063828 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:14.063833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:14.063836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:14.064207 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:14.064957 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:14.064966 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:14.065514 (+ 548us) client_negotiation.cc:769] Sending connection context
0927 22:57:14.065904 (+ 390us) client_negotiation.cc:240] Negotiation successful
0927 22:57:14.066519 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":446,"threads_started":1}
I20260927 22:57:14.095690 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.075680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:60793 (local address 127.4.74.190:39797)
0927 22:57:14.075918 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:14.075926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:14.076711 (+ 785us) server_negotiation.cc:407] Connection header received
0927 22:57:14.077029 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:14.077037 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:14.077150 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:14.077283 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:14.078414 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.079728 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.081379 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.082161 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.082319 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.086343 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:14.086355 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:14.086362 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:14.086423 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:14.091047 (+ 4624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:14.092494 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:14.092504 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:14.092509 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:14.092726 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:14.093305 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:14.093312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:14.093316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:14.093940 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:14.094129 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:14.094836 (+ 707us) server_negotiation.cc:299] Negotiation successful
0927 22:57:14.095188 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 22:57:14.096854 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.075616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39797 (local address 127.4.74.130:60793)
0927 22:57:14.076529 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:14.076562 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:14.076815 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:14.077353 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:14.077358 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:14.077891 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:14.078250 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.078259 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.079960 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.079969 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.081202 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.081216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.082478 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.082489 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.082792 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.083849 (+ 1057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:14.083882 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:14.086006 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:14.091296 (+ 5290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:14.091306 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:14.091313 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:14.092293 (+ 980us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:14.092900 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:14.092908 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:14.092913 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:14.093115 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:14.094112 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:14.094119 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:14.094436 (+ 317us) client_negotiation.cc:769] Sending connection context
0927 22:57:14.094665 (+ 229us) client_negotiation.cc:240] Negotiation successful
0927 22:57:14.095240 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":405,"threads_started":1}
I20260927 22:57:14.101471 5733 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:32837
I20260927 22:57:14.101529 5868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:32837 every 8 connection(s)
I20260927 22:57:14.103273 5733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:57:14.108193 5869 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39797
I20260927 22:57:14.108538 5869 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:14.108629 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 5733
I20260927 22:57:14.108836 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:57:14.109509 4394 external_mini_cluster.cc:1428] Setting key 66129924f675c52aa255229a57645921
I20260927 22:57:14.109598 5869 heartbeater.cc:511] Master 127.4.74.190:39797 requested a full tablet report, sending...
I20260927 22:57:14.111809 5521 ts_manager.cc:194] Registered new tserver with Master: cc4dbbf5fdef4b07a979f2b17f0c808e (127.4.74.130:32837)
I20260927 22:57:14.113220 5521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:48271
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:57:14.160874 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39797
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:39461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:57:14.527952 5729 heartbeater.cc:503] Master 127.4.74.190:39797 was elected leader, sending a full tablet report...
W20260927 22:57:14.572886 5873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:14.573262 5873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:14.573374 5873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:14.581813 5873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:14.581975 5873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:14.582072 5873 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 22:57:14.582166 5873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:57:14.593940 5873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:39461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.74.190:39797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:14.595824 5873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:14.597659 5873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:14.613794 5878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:14.614534 5881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:14.614557 5879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:14.615124 5873 server_base.cc:1034] running on GCE node
I20260927 22:57:14.615995 5873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:14.617244 5873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:14.618510 5873 hybrid_clock.cc:648] HybridClock initialized: now 1790549834618425 us; error 56 us; skew 500 ppm
I20260927 22:57:14.618940 5873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:14 dist-test-slave-5pd3 krb5kdc[5477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549834, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:14.622867 5873 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:57:14.625051 5873 webserver.cc:460] Webserver started at http://127.4.74.131:35883/ using document root <none> and password file <none>
I20260927 22:57:14.625957 5873 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:14.626055 5873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:14.626376 5873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:14.630407 5873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "991e8dc6f4d448748f552ae87e9cc858"
format_stamp: "Formatted at 2026-09-27 22:57:14 on dist-test-slave-5pd3"
server_key: "7408d00c20179f0f32f3417fc42f9de2"
server_key_iv: "412d02929694a829cc53e01e18fbefee"
server_key_version: "encryptionkey@0"
I20260927 22:57:14.631448 5873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "991e8dc6f4d448748f552ae87e9cc858"
format_stamp: "Formatted at 2026-09-27 22:57:14 on dist-test-slave-5pd3"
server_key: "7408d00c20179f0f32f3417fc42f9de2"
server_key_iv: "412d02929694a829cc53e01e18fbefee"
server_key_version: "encryptionkey@0"
I20260927 22:57:14.638813 5873 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260927 22:57:14.644644 5888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:14.646903 5873 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260927 22:57:14.647150 5873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "991e8dc6f4d448748f552ae87e9cc858"
format_stamp: "Formatted at 2026-09-27 22:57:14 on dist-test-slave-5pd3"
server_key: "7408d00c20179f0f32f3417fc42f9de2"
server_key_iv: "412d02929694a829cc53e01e18fbefee"
server_key_version: "encryptionkey@0"
I20260927 22:57:14.647413 5873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:14.669062 5873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:14.674713 5873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:14.675010 5873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:14.678491 5873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:14.678592 5873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:14.678787 5873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:14.678896 5873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 22:57:14 dist-test-slave-5pd3 krb5kdc[5477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549834, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:14.712098 5897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.679361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:46917 (local address 127.4.74.190:39797)
0927 22:57:14.681498 (+ 2137us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:14.681502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:14.682094 (+ 592us) server_negotiation.cc:407] Connection header received
0927 22:57:14.685511 (+ 3417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:14.685519 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:14.685677 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:14.685822 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:14.689434 (+ 3612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.690683 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.694012 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.694814 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.695036 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.702198 (+ 7162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:14.702207 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:14.702215 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:14.702283 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:14.706500 (+ 4217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:14.708043 (+ 1543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:14.708050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:14.708055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:14.708357 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:14.709156 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:14.709162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:14.709167 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:14.709691 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:14.709881 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:14.711194 (+ 1313us) server_negotiation.cc:299] Negotiation successful
0927 22:57:14.711516 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2010,"thread_start_us":1909,"threads_started":1}
I20260927 22:57:14.714203 5898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.680489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39797 (local address 127.4.74.131:46917)
0927 22:57:14.681813 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:14.681925 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:14.684672 (+ 2747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:14.686434 (+ 1762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:14.686454 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:14.687816 (+ 1362us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:14.689125 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.689171 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.690954 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.690959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.693763 (+ 2804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.693787 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.695257 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.695274 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.695836 (+ 562us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.696945 (+ 1109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:14.696989 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:14.701786 (+ 4797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:14.706724 (+ 4938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:14.706745 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:14.706808 (+ 63us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:14.707856 (+ 1048us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:14.708518 (+ 662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:14.708525 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:14.708536 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:14.709008 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:14.709844 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:14.709855 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:14.710624 (+ 769us) client_negotiation.cc:769] Sending connection context
0927 22:57:14.711060 (+ 436us) client_negotiation.cc:240] Negotiation successful
0927 22:57:14.711921 (+ 861us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"thread_start_us":602,"threads_started":1}
I20260927 22:57:14.739632 5873 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:40491
I20260927 22:57:14.739696 6008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:40491 every 8 connection(s)
I20260927 22:57:14.741446 5873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:57:14.743042 5897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.723597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:45617 (local address 127.4.74.190:39797)
0927 22:57:14.723880 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:14.723887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:14.727204 (+ 3317us) server_negotiation.cc:407] Connection header received
0927 22:57:14.727615 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:14.727622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:14.727729 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:14.727794 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:14.729017 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.730568 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.732650 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.733167 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.733311 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.735479 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:14.735484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:14.735488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:14.735522 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:14.739899 (+ 4377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:14.740750 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:14.740757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:14.740761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:14.740919 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:14.741316 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:14.741321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:14.741324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:14.741742 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:14.741902 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:14.742363 (+ 461us) server_negotiation.cc:299] Negotiation successful
0927 22:57:14.742632 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260927 22:57:14.743238 5991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.723377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39797 (local address 127.4.74.131:45617)
0927 22:57:14.727027 (+ 3650us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:14.727057 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:14.727379 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:14.727996 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:14.728000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:14.728506 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:14.728857 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.728867 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.730805 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.730812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.732501 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.732514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.733405 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.733412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.733589 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.734389 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:14.734413 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:14.735236 (+ 823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:14.740133 (+ 4897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:14.740139 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:14.740143 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:14.740610 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:14.741041 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:14.741047 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:14.741051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:14.741194 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:14.741858 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:14.741863 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:14.742090 (+ 227us) client_negotiation.cc:769] Sending connection context
0927 22:57:14.742269 (+ 179us) client_negotiation.cc:240] Negotiation successful
0927 22:57:14.742691 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3390,"thread_start_us":2834,"threads_started":1}
I20260927 22:57:14.744627 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 5873
I20260927 22:57:14.744861 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:57:14.744459 6009 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39797
I20260927 22:57:14.745330 6009 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:14.745522 4394 external_mini_cluster.cc:1428] Setting key 5e22fa260a3db52518d96b55ee05b7c8
I20260927 22:57:14.746433 6009 heartbeater.cc:511] Master 127.4.74.190:39797 requested a full tablet report, sending...
I20260927 22:57:14.748231 5522 ts_manager.cc:194] Registered new tserver with Master: 991e8dc6f4d448748f552ae87e9cc858 (127.4.74.131:40491)
I20260927 22:57:14.749550 5522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:46917
I20260927 22:57:14.749679 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 22:57:14 dist-test-slave-5pd3 krb5kdc[5477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549834, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 22:57:14 dist-test-slave-5pd3 krb5kdc[5477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549834, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:14.805598 5897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.788569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.4.74.190:39797)
0927 22:57:14.788910 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:14.788963 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:14.789334 (+ 371us) server_negotiation.cc:407] Connection header received
0927 22:57:14.789660 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:14.789669 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:14.789759 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:14.789879 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:14.791683 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.792514 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.793939 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.794391 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.794534 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.799421 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:14.799428 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:14.799432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:14.799478 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:14.802451 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:14.803324 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:14.803335 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:14.803337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:14.803478 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:14.803933 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:14.803940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:14.803943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:14.804262 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:14.804573 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:14.804914 (+ 341us) server_negotiation.cc:299] Negotiation successful
0927 22:57:14.805204 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260927 22:57:14.811091 5522 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:57:14.818948 5522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:14.861361 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.835802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:40491 (local address 127.0.0.1:49682)
0927 22:57:14.843920 (+ 8118us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:14.843981 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:14.844215 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:14.850002 (+ 5787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:14.850016 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:14.850071 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:14.850673 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.850691 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.853549 (+ 2858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.853559 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.857042 (+ 3483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.857056 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.858691 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.858702 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.860423 (+ 1721us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.860505 (+ 82us) client_negotiation.cc:769] Sending connection context
0927 22:57:14.860671 (+ 166us) client_negotiation.cc:240] Negotiation successful
0927 22:57:14.860897 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7781,"mutex_wait_us":114,"thread_start_us":17004,"threads_started":1}
I20260927 22:57:14.861467 6022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.835997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.4.74.131:40491)
0927 22:57:14.848619 (+ 12622us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:14.848631 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:14.848676 (+ 45us) server_negotiation.cc:407] Connection header received
0927 22:57:14.848814 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:14.848824 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:14.849187 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:14.849494 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:14.851014 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.853259 (+ 2245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.857253 (+ 3994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.858387 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.858587 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.858924 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:14.860773 (+ 1849us) server_negotiation.cc:299] Negotiation successful
0927 22:57:14.861031 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12384,"thread_start_us":12781,"threads_started":1}
I20260927 22:57:14.863688 6025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.836394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59552 (local address 127.4.74.129:37123)
0927 22:57:14.847791 (+ 11397us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:14.847802 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:14.852759 (+ 4957us) server_negotiation.cc:407] Connection header received
0927 22:57:14.853574 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:14.853592 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:14.854047 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:14.854320 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:14.855573 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.857191 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.860611 (+ 3420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.862362 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.862559 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.862824 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:14.863081 (+ 257us) server_negotiation.cc:299] Negotiation successful
0927 22:57:14.863298 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11155,"thread_start_us":11393,"threads_started":1}
I20260927 22:57:14.864494 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.837378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:32837 (local address 127.0.0.1:37850)
0927 22:57:14.854752 (+ 17374us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:14.854783 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:14.854929 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:14.856150 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:14.856154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:14.856182 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:14.856613 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.856624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.858500 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.858508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.861358 (+ 2850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.861374 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.863535 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.863566 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.863778 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.863814 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 22:57:14.863923 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 22:57:14.864068 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17197,"mutex_wait_us":67,"thread_start_us":17016,"threads_started":1}
I20260927 22:57:14.865187 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.837660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37850 (local address 127.4.74.130:32837)
0927 22:57:14.855436 (+ 17776us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:14.855448 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:14.855480 (+ 32us) server_negotiation.cc:407] Connection header received
0927 22:57:14.855615 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:14.855625 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:14.855914 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:14.856143 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:14.856832 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.858330 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.861570 (+ 3240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.862951 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.863138 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.863361 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:14.864800 (+ 1439us) server_negotiation.cc:299] Negotiation successful
0927 22:57:14.864951 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17608,"thread_start_us":18616,"threads_started":1}
I20260927 22:57:14.866842 5939 tablet_service.cc:1467] Processing CreateTablet for tablet f3b4fe2ccf60493699ee51ab0e96e832 (DEFAULT_TABLE table=test-table [id=6c5f00aad507426086acdfdf67e44dfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:14.868156 6024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:14.836140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37123 (local address 127.0.0.1:59552)
0927 22:57:14.852523 (+ 16383us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:14.852564 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:14.853278 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:14.854662 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:14.854668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:14.854702 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:14.855329 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.855341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.857354 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.857360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.860404 (+ 3044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:14.860420 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:14.862629 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:14.862636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:14.862833 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:14.862878 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 22:57:14.863014 (+ 136us) client_negotiation.cc:240] Negotiation successful
0927 22:57:14.867764 (+ 4750us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16153,"mutex_wait_us":136,"thread_start_us":7269,"threads_started":1}
I20260927 22:57:14.869143 5939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b4fe2ccf60493699ee51ab0e96e832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:14.869143 5799 tablet_service.cc:1467] Processing CreateTablet for tablet f3b4fe2ccf60493699ee51ab0e96e832 (DEFAULT_TABLE table=test-table [id=6c5f00aad507426086acdfdf67e44dfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:14.871605 5799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b4fe2ccf60493699ee51ab0e96e832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:14.872951 5659 tablet_service.cc:1467] Processing CreateTablet for tablet f3b4fe2ccf60493699ee51ab0e96e832 (DEFAULT_TABLE table=test-table [id=6c5f00aad507426086acdfdf67e44dfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:14.874440 5659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b4fe2ccf60493699ee51ab0e96e832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:14.885362 6028 tablet_bootstrap.cc:492] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: Bootstrap starting.
I20260927 22:57:14.890154 6028 tablet_bootstrap.cc:654] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:14.890763 6029 tablet_bootstrap.cc:492] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: Bootstrap starting.
I20260927 22:57:14.891837 6030 tablet_bootstrap.cc:492] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: Bootstrap starting.
I20260927 22:57:14.892673 6028 log.cc:824] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:14.896150 6030 tablet_bootstrap.cc:654] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:14.896140 6029 tablet_bootstrap.cc:654] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:14.897455 6030 log.cc:824] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:14.897791 6028 tablet_bootstrap.cc:492] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: No bootstrap required, opened a new log
I20260927 22:57:14.898030 6029 log.cc:824] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:14.898211 6028 ts_tablet_manager.cc:1397] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.002s
I20260927 22:57:14.900979 6030 tablet_bootstrap.cc:492] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: No bootstrap required, opened a new log
I20260927 22:57:14.901315 6030 ts_tablet_manager.cc:1397] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260927 22:57:14.902784 6029 tablet_bootstrap.cc:492] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: No bootstrap required, opened a new log
I20260927 22:57:14.903148 6029 ts_tablet_manager.cc:1397] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.001s
I20260927 22:57:14.909934 6030 raft_consensus.cc:348] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } }
I20260927 22:57:14.910401 6030 raft_consensus.cc:374] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:14.910518 6030 raft_consensus.cc:729] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4dbbf5fdef4b07a979f2b17f0c808e, State: Initialized, Role: FOLLOWER
I20260927 22:57:14.910538 6028 raft_consensus.cc:348] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } }
I20260927 22:57:14.911142 6028 raft_consensus.cc:374] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:14.911254 6028 raft_consensus.cc:729] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991e8dc6f4d448748f552ae87e9cc858, State: Initialized, Role: FOLLOWER
I20260927 22:57:14.911345 6030 consensus_queue.cc:260] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } }
I20260927 22:57:14.912029 6028 consensus_queue.cc:260] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } }
I20260927 22:57:14.912619 5869 heartbeater.cc:503] Master 127.4.74.190:39797 was elected leader, sending a full tablet report...
I20260927 22:57:14.913410 6030 ts_tablet_manager.cc:1428] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260927 22:57:14.915864 6029 raft_consensus.cc:348] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } }
I20260927 22:57:14.916747 6029 raft_consensus.cc:374] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:14.916930 6029 raft_consensus.cc:729] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00ea55ce1714046a8a23fd8cd77a8f7, State: Initialized, Role: FOLLOWER
I20260927 22:57:14.918243 6029 consensus_queue.cc:260] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } }
I20260927 22:57:14.919996 6009 heartbeater.cc:503] Master 127.4.74.190:39797 was elected leader, sending a full tablet report...
I20260927 22:57:14.921293 6028 ts_tablet_manager.cc:1428] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: Time spent starting tablet: real 0.023s user 0.009s sys 0.012s
I20260927 22:57:14.923311 6029 ts_tablet_manager.cc:1428] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: Time spent starting tablet: real 0.020s user 0.009s sys 0.011s
W20260927 22:57:14.995731 6010 tablet.cc:2369] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 22:57:15.018472 5730 tablet.cc:2369] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 22:57:15.107591 5870 tablet.cc:2369] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:57:15.136102 6036 raft_consensus.cc:482] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:15.136674 6036 raft_consensus.cc:504] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } }
I20260927 22:57:15.138908 6036 leader_election.cc:288] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 991e8dc6f4d448748f552ae87e9cc858 (127.4.74.131:40491), cc4dbbf5fdef4b07a979f2b17f0c808e (127.4.74.130:32837)
I20260927 22:57:15.147778 6022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.139503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:53355 (local address 127.4.74.131:40491)
0927 22:57:15.139772 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:15.139781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:15.140550 (+ 769us) server_negotiation.cc:407] Connection header received
0927 22:57:15.140830 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:15.140837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:15.140936 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:15.141074 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:15.142147 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.143361 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.145808 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.146636 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.146860 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.146969 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:15.147272 (+ 303us) server_negotiation.cc:299] Negotiation successful
0927 22:57:15.147435 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260927 22:57:15.147799 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.139503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:45457 (local address 127.4.74.130:32837)
0927 22:57:15.139844 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:15.139854 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:15.140546 (+ 692us) server_negotiation.cc:407] Connection header received
0927 22:57:15.140830 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:15.140837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:15.140936 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:15.141074 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:15.142131 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.143332 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.145844 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.146638 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.146876 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.146970 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:15.147328 (+ 358us) server_negotiation.cc:299] Negotiation successful
0927 22:57:15.147464 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260927 22:57:15.148003 6037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.139367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:32837 (local address 127.4.74.129:45457)
0927 22:57:15.140329 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:15.140354 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:15.140583 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:15.141137 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:15.141143 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:15.141206 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:15.141870 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.141890 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.143648 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.143653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.145613 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.145631 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.146989 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.146998 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.147153 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.147181 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 22:57:15.147302 (+ 121us) client_negotiation.cc:240] Negotiation successful
0927 22:57:15.147476 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":54,"spinlock_wait_cycles":66176,"thread_start_us":495,"threads_started":1}
I20260927 22:57:15.148514 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.139426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:40491 (local address 127.4.74.129:53355)
0927 22:57:15.140329 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:15.140354 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:15.140583 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:15.141137 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:15.141143 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:15.141204 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:15.141909 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.141918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.143632 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.143637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.145608 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.145622 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.146918 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.146924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.147094 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.147122 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 22:57:15.147243 (+ 121us) client_negotiation.cc:240] Negotiation successful
0927 22:57:15.147409 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":150,"spinlock_wait_cycles":44544,"thread_start_us":603,"threads_started":1}
I20260927 22:57:15.149255 5824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b4fe2ccf60493699ee51ab0e96e832" candidate_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" is_pre_election: true
I20260927 22:57:15.149631 5959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b4fe2ccf60493699ee51ab0e96e832" candidate_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991e8dc6f4d448748f552ae87e9cc858" is_pre_election: true
I20260927 22:57:15.149905 5824 raft_consensus.cc:2454] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e00ea55ce1714046a8a23fd8cd77a8f7 in term 0.
I20260927 22:57:15.150187 5959 raft_consensus.cc:2454] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e00ea55ce1714046a8a23fd8cd77a8f7 in term 0.
I20260927 22:57:15.150802 5611 leader_election.cc:302] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 991e8dc6f4d448748f552ae87e9cc858, cc4dbbf5fdef4b07a979f2b17f0c808e, e00ea55ce1714046a8a23fd8cd77a8f7; no voters:
I20260927 22:57:15.151424 6036 raft_consensus.cc:2781] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:15.151669 6036 raft_consensus.cc:482] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:15.151821 6036 raft_consensus.cc:3037] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:15.154620 6036 raft_consensus.cc:504] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } }
I20260927 22:57:15.155442 6036 leader_election.cc:288] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [CANDIDATE]: Term 1 election: Requested vote from peers 991e8dc6f4d448748f552ae87e9cc858 (127.4.74.131:40491), cc4dbbf5fdef4b07a979f2b17f0c808e (127.4.74.130:32837)
I20260927 22:57:15.155977 5959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b4fe2ccf60493699ee51ab0e96e832" candidate_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991e8dc6f4d448748f552ae87e9cc858"
I20260927 22:57:15.156224 5959 raft_consensus.cc:3037] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:15.156293 5824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b4fe2ccf60493699ee51ab0e96e832" candidate_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e"
I20260927 22:57:15.156631 5824 raft_consensus.cc:3037] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:15.158704 5959 raft_consensus.cc:2454] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e00ea55ce1714046a8a23fd8cd77a8f7 in term 1.
I20260927 22:57:15.159250 5610 leader_election.cc:302] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 991e8dc6f4d448748f552ae87e9cc858, e00ea55ce1714046a8a23fd8cd77a8f7; no voters:
I20260927 22:57:15.159634 6036 raft_consensus.cc:2781] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:15.159528 5824 raft_consensus.cc:2454] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e00ea55ce1714046a8a23fd8cd77a8f7 in term 1.
I20260927 22:57:15.160211 6036 raft_consensus.cc:686] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 LEADER]: Becoming Leader. State: Replica: e00ea55ce1714046a8a23fd8cd77a8f7, State: Running, Role: LEADER
I20260927 22:57:15.160802 6036 consensus_queue.cc:237] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } }
I20260927 22:57:15.166676 5521 catalog_manager.cc:5495] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e00ea55ce1714046a8a23fd8cd77a8f7 (127.4.74.129). New cstate: current_term: 1 leader_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } health_report { overall_health: UNKNOWN } } }
I20260927 22:57:15.249200 6025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.242457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55584 (local address 127.4.74.129:37123)
0927 22:57:15.242769 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:15.242778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:15.242934 (+ 156us) server_negotiation.cc:407] Connection header received
0927 22:57:15.243373 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:15.243379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:15.243496 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:15.243725 (+ 229us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:15.244141 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.244923 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.246445 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.247039 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.247193 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.247872 (+ 679us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:15.248471 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:15.248656 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:15.248799 (+ 143us) server_negotiation.cc:299] Negotiation successful
0927 22:57:15.248890 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260927 22:57:15.260330 5959 raft_consensus.cc:1261] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858 [term 1 FOLLOWER]: Refusing update from remote peer e00ea55ce1714046a8a23fd8cd77a8f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:15.260330 5824 raft_consensus.cc:1261] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 1 FOLLOWER]: Refusing update from remote peer e00ea55ce1714046a8a23fd8cd77a8f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:15.261740 6039 consensus_queue.cc:1035] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:15.262226 6036 consensus_queue.cc:1035] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:15.303467 5522 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:15.303901 5522 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:15.308405 5522 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec: Sending DeleteTablet for 3 replicas of tablet f3b4fe2ccf60493699ee51ab0e96e832
I20260927 22:57:15.310060 5659 tablet_service.cc:1514] Processing DeleteTablet for tablet f3b4fe2ccf60493699ee51ab0e96e832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:15 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:59552
I20260927 22:57:15.310074 5939 tablet_service.cc:1514] Processing DeleteTablet for tablet f3b4fe2ccf60493699ee51ab0e96e832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:15 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:49682
I20260927 22:57:15.310940 5799 tablet_service.cc:1514] Processing DeleteTablet for tablet f3b4fe2ccf60493699ee51ab0e96e832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:15 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:37850
I20260927 22:57:15.311317 6052 tablet_replica.cc:330] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: stopping tablet replica
I20260927 22:57:15.312042 6052 raft_consensus.cc:2229] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:15.312834 6052 raft_consensus.cc:2258] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:15.313954 6054 tablet_replica.cc:330] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: stopping tablet replica
I20260927 22:57:15.314587 6054 raft_consensus.cc:2229] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:15.315255 6054 raft_consensus.cc:2258] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:15.315121 5522 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:57:15.315987 5522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:15.318576 6052 ts_tablet_manager.cc:1905] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:15.327353 5658 tablet_service.cc:1467] Processing CreateTablet for tablet fc2a37c6883d44ada112a57f8f29acbf (DEFAULT_TABLE table=test-table [id=6e2998a6d9c84483af3f6e3e5877dbe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:15.328679 5658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2a37c6883d44ada112a57f8f29acbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:15.330793 6054 ts_tablet_manager.cc:1905] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:15.332293 5799 tablet_service.cc:1467] Processing CreateTablet for tablet fc2a37c6883d44ada112a57f8f29acbf (DEFAULT_TABLE table=test-table [id=6e2998a6d9c84483af3f6e3e5877dbe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:15.332857 5799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2a37c6883d44ada112a57f8f29acbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:15.336869 5939 tablet_service.cc:1467] Processing CreateTablet for tablet fc2a37c6883d44ada112a57f8f29acbf (DEFAULT_TABLE table=test-table [id=6e2998a6d9c84483af3f6e3e5877dbe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:15.337442 5939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2a37c6883d44ada112a57f8f29acbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:15.343149 6052 ts_tablet_manager.cc:1918] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:57:15.343511 6052 log.cc:1196] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/f3b4fe2ccf60493699ee51ab0e96e832
I20260927 22:57:15.344285 6052 ts_tablet_manager.cc:1939] T f3b4fe2ccf60493699ee51ab0e96e832 P 991e8dc6f4d448748f552ae87e9cc858: Deleting consensus metadata
I20260927 22:57:15.347461 5507 catalog_manager.cc:4841] TS 991e8dc6f4d448748f552ae87e9cc858 (127.4.74.131:40491): tablet f3b4fe2ccf60493699ee51ab0e96e832 (table test-table [id=6c5f00aad507426086acdfdf67e44dfd]) successfully deleted
I20260927 22:57:15.353405 6054 ts_tablet_manager.cc:1918] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:57:15.353760 6054 log.cc:1196] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/f3b4fe2ccf60493699ee51ab0e96e832
I20260927 22:57:15.354539 6054 ts_tablet_manager.cc:1939] T f3b4fe2ccf60493699ee51ab0e96e832 P cc4dbbf5fdef4b07a979f2b17f0c808e: Deleting consensus metadata
I20260927 22:57:15.358605 5508 catalog_manager.cc:4841] TS cc4dbbf5fdef4b07a979f2b17f0c808e (127.4.74.130:32837): tablet f3b4fe2ccf60493699ee51ab0e96e832 (table test-table [id=6c5f00aad507426086acdfdf67e44dfd]) successfully deleted
I20260927 22:57:15.359172 6029 tablet_bootstrap.cc:492] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7: Bootstrap starting.
I20260927 22:57:15.361233 6029 tablet_bootstrap.cc:654] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:15.363179 6030 tablet_bootstrap.cc:492] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e: Bootstrap starting.
I20260927 22:57:15.363926 6028 tablet_bootstrap.cc:492] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858: Bootstrap starting.
I20260927 22:57:15.364934 6053 tablet_replica.cc:330] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: stopping tablet replica
I20260927 22:57:15.365119 6030 tablet_bootstrap.cc:654] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:15.365537 6053 raft_consensus.cc:2229] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:57:15.365935 6028 tablet_bootstrap.cc:654] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:15.366396 6053 raft_consensus.cc:2258] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:15.369336 6030 tablet_bootstrap.cc:492] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e: No bootstrap required, opened a new log
I20260927 22:57:15.369500 6030 ts_tablet_manager.cc:1397] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260927 22:57:15.369635 6053 ts_tablet_manager.cc:1905] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:15.370146 6029 tablet_bootstrap.cc:492] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7: No bootstrap required, opened a new log
I20260927 22:57:15.370374 6029 ts_tablet_manager.cc:1397] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260927 22:57:15.370357 6030 raft_consensus.cc:348] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.370596 6030 raft_consensus.cc:374] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:15.370712 6030 raft_consensus.cc:729] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4dbbf5fdef4b07a979f2b17f0c808e, State: Initialized, Role: FOLLOWER
I20260927 22:57:15.371011 6030 consensus_queue.cc:260] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.371747 6029 raft_consensus.cc:348] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.372006 6029 raft_consensus.cc:374] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:15.372114 6029 raft_consensus.cc:729] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00ea55ce1714046a8a23fd8cd77a8f7, State: Initialized, Role: FOLLOWER
I20260927 22:57:15.372457 6029 consensus_queue.cc:260] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.373188 6029 ts_tablet_manager.cc:1428] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 22:57:15.374462 6028 tablet_bootstrap.cc:492] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858: No bootstrap required, opened a new log
I20260927 22:57:15.374699 6028 ts_tablet_manager.cc:1397] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260927 22:57:15.375664 6028 raft_consensus.cc:348] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.375977 6028 raft_consensus.cc:374] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:15.376125 6028 raft_consensus.cc:729] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991e8dc6f4d448748f552ae87e9cc858, State: Initialized, Role: FOLLOWER
I20260927 22:57:15.376478 6028 consensus_queue.cc:260] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.377259 6030 ts_tablet_manager.cc:1428] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260927 22:57:15.378154 6028 ts_tablet_manager.cc:1428] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 22:57:15.381986 6053 ts_tablet_manager.cc:1918] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:57:15.382247 6053 log.cc:1196] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/f3b4fe2ccf60493699ee51ab0e96e832
I20260927 22:57:15.382928 6053 ts_tablet_manager.cc:1939] T f3b4fe2ccf60493699ee51ab0e96e832 P e00ea55ce1714046a8a23fd8cd77a8f7: Deleting consensus metadata
I20260927 22:57:15.385234 5506 catalog_manager.cc:4841] TS e00ea55ce1714046a8a23fd8cd77a8f7 (127.4.74.129:37123): tablet f3b4fe2ccf60493699ee51ab0e96e832 (table test-table [id=6c5f00aad507426086acdfdf67e44dfd]) successfully deleted
I20260927 22:57:15.456058 6047 raft_consensus.cc:482] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:15.456393 6047 raft_consensus.cc:504] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.457454 6047 leader_election.cc:288] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc4dbbf5fdef4b07a979f2b17f0c808e (127.4.74.130:32837), 991e8dc6f4d448748f552ae87e9cc858 (127.4.74.131:40491)
I20260927 22:57:15.458173 5824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2a37c6883d44ada112a57f8f29acbf" candidate_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" is_pre_election: true
I20260927 22:57:15.458473 5824 raft_consensus.cc:2454] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e00ea55ce1714046a8a23fd8cd77a8f7 in term 0.
I20260927 22:57:15.458424 5959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2a37c6883d44ada112a57f8f29acbf" candidate_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991e8dc6f4d448748f552ae87e9cc858" is_pre_election: true
I20260927 22:57:15.458746 5959 raft_consensus.cc:2454] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e00ea55ce1714046a8a23fd8cd77a8f7 in term 0.
I20260927 22:57:15.458966 5611 leader_election.cc:302] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4dbbf5fdef4b07a979f2b17f0c808e, e00ea55ce1714046a8a23fd8cd77a8f7; no voters:
I20260927 22:57:15.459363 6047 raft_consensus.cc:2781] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:15.459578 6047 raft_consensus.cc:482] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:15.459715 6047 raft_consensus.cc:3037] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:15.462666 6047 raft_consensus.cc:504] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.464000 6047 leader_election.cc:288] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [CANDIDATE]: Term 1 election: Requested vote from peers cc4dbbf5fdef4b07a979f2b17f0c808e (127.4.74.130:32837), 991e8dc6f4d448748f552ae87e9cc858 (127.4.74.131:40491)
I20260927 22:57:15.464008 5959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2a37c6883d44ada112a57f8f29acbf" candidate_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991e8dc6f4d448748f552ae87e9cc858"
I20260927 22:57:15.464124 5824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2a37c6883d44ada112a57f8f29acbf" candidate_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e"
I20260927 22:57:15.464279 5959 raft_consensus.cc:3037] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:15.464325 5824 raft_consensus.cc:3037] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:15.467337 5824 raft_consensus.cc:2454] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e00ea55ce1714046a8a23fd8cd77a8f7 in term 1.
I20260927 22:57:15.467337 5959 raft_consensus.cc:2454] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e00ea55ce1714046a8a23fd8cd77a8f7 in term 1.
I20260927 22:57:15.468008 5611 leader_election.cc:302] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4dbbf5fdef4b07a979f2b17f0c808e, e00ea55ce1714046a8a23fd8cd77a8f7; no voters:
I20260927 22:57:15.468544 6047 raft_consensus.cc:2781] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:15.468787 6047 raft_consensus.cc:686] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 LEADER]: Becoming Leader. State: Replica: e00ea55ce1714046a8a23fd8cd77a8f7, State: Running, Role: LEADER
I20260927 22:57:15.469163 6047 consensus_queue.cc:237] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.472952 5521 catalog_manager.cc:5495] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e00ea55ce1714046a8a23fd8cd77a8f7 (127.4.74.129). New cstate: current_term: 1 leader_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } health_report { overall_health: UNKNOWN } } }
Sep 27 22:57:15 dist-test-slave-5pd3 krb5kdc[5477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549832, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:15.552798 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.518898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55794 (local address 127.4.74.190:39797)
0927 22:57:15.534696 (+ 15798us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:15.534703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:15.534742 (+ 39us) server_negotiation.cc:407] Connection header received
0927 22:57:15.534825 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:15.534833 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:15.534929 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:15.535103 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:15.536718 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.537885 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.539906 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.540697 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.540867 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.545819 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:15.545829 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:15.545834 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:15.545880 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:15.548984 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:15.550168 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:15.550195 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:15.550199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:15.550367 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:15.550829 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:15.550832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:15.550835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:15.551159 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:15.551309 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:15.551971 (+ 662us) server_negotiation.cc:299] Negotiation successful
0927 22:57:15.552323 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":15598,"thread_start_us":15791,"threads_started":1}
I20260927 22:57:15.553082 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.518852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39797 (local address 127.0.0.1:55794)
0927 22:57:15.534105 (+ 15253us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:15.534134 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:15.534324 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:15.535208 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:15.535213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:15.536205 (+ 992us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:15.536509 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.536522 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.538081 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.538087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.539731 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.539748 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.540927 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.540934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.541139 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.542027 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:15.542050 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:15.545501 (+ 3451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:15.549178 (+ 3677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:15.549199 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:15.549209 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:15.549989 (+ 780us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:15.550493 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:15.550499 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:15.550503 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:15.550691 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:15.551323 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:15.551333 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:15.551641 (+ 308us) client_negotiation.cc:769] Sending connection context
0927 22:57:15.551863 (+ 222us) client_negotiation.cc:240] Negotiation successful
0927 22:57:15.552540 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15043,"mutex_wait_us":210,"thread_start_us":15573,"threads_started":1}
I20260927 22:57:15.560003 5521 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:55794:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 22:57:15.561218 5521 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:15.569960 5799 tablet_service.cc:1467] Processing CreateTablet for tablet da2fe6c658554ae499906fea826f8372 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cbcabfc5ae54e2e939f690d3485ccee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 22:57:15.569960 5658 tablet_service.cc:1467] Processing CreateTablet for tablet da2fe6c658554ae499906fea826f8372 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cbcabfc5ae54e2e939f690d3485ccee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 22:57:15.570658 5658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2fe6c658554ae499906fea826f8372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:15.570691 5799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2fe6c658554ae499906fea826f8372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:15.572091 5939 tablet_service.cc:1467] Processing CreateTablet for tablet da2fe6c658554ae499906fea826f8372 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cbcabfc5ae54e2e939f690d3485ccee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 22:57:15.572695 5939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2fe6c658554ae499906fea826f8372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:15.579073 6028 tablet_bootstrap.cc:492] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: Bootstrap starting.
I20260927 22:57:15.579092 6030 tablet_bootstrap.cc:492] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: Bootstrap starting.
I20260927 22:57:15.580497 6029 tablet_bootstrap.cc:492] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: Bootstrap starting.
I20260927 22:57:15.580909 6028 tablet_bootstrap.cc:654] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:15.581118 6030 tablet_bootstrap.cc:654] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:15.582537 6029 tablet_bootstrap.cc:654] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:15.589915 6028 tablet_bootstrap.cc:492] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: No bootstrap required, opened a new log
I20260927 22:57:15.590225 6028 ts_tablet_manager.cc:1397] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260927 22:57:15.590830 6030 tablet_bootstrap.cc:492] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: No bootstrap required, opened a new log
I20260927 22:57:15.591068 6030 ts_tablet_manager.cc:1397] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260927 22:57:15.591346 6028 raft_consensus.cc:348] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.591636 6028 raft_consensus.cc:374] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:15.591737 6028 raft_consensus.cc:729] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991e8dc6f4d448748f552ae87e9cc858, State: Initialized, Role: FOLLOWER
I20260927 22:57:15.591991 6030 raft_consensus.cc:348] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.592075 6028 consensus_queue.cc:260] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.592397 6030 raft_consensus.cc:374] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:15.592533 6030 raft_consensus.cc:729] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4dbbf5fdef4b07a979f2b17f0c808e, State: Initialized, Role: FOLLOWER
I20260927 22:57:15.592617 6035 tablet_replica.cc:439] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } }
I20260927 22:57:15.592993 6035 tablet_replica.cc:442] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:15.592890 6030 consensus_queue.cc:260] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.593523 6030 ts_tablet_manager.cc:1428] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260927 22:57:15.594007 6034 tablet_replica.cc:439] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } }
I20260927 22:57:15.594523 6034 tablet_replica.cc:442] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:15.595356 6028 ts_tablet_manager.cc:1428] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260927 22:57:15.596462 6029 tablet_bootstrap.cc:492] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: No bootstrap required, opened a new log
I20260927 22:57:15.596756 6029 ts_tablet_manager.cc:1397] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.000s
I20260927 22:57:15.597733 6029 raft_consensus.cc:348] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.598124 6029 raft_consensus.cc:374] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:15.598299 6029 raft_consensus.cc:729] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00ea55ce1714046a8a23fd8cd77a8f7, State: Initialized, Role: FOLLOWER
I20260927 22:57:15.598721 6029 consensus_queue.cc:260] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.599583 6029 ts_tablet_manager.cc:1428] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260927 22:57:15.599366 6047 tablet_replica.cc:439] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } }
I20260927 22:57:15.600600 6047 tablet_replica.cc:442] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:15.858107 6034 raft_consensus.cc:482] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:15.858433 6034 raft_consensus.cc:504] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.860412 6034 leader_election.cc:288] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e00ea55ce1714046a8a23fd8cd77a8f7 (127.4.74.129:37123), 991e8dc6f4d448748f552ae87e9cc858 (127.4.74.131:40491)
I20260927 22:57:15.881632 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.861170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:36263 (local address 127.4.74.129:37123)
0927 22:57:15.872888 (+ 11718us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:15.872896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:15.873394 (+ 498us) server_negotiation.cc:407] Connection header received
0927 22:57:15.873621 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:15.873628 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:15.873720 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:15.873871 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:15.874740 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.875892 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.879014 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.880358 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.880508 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.880586 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:15.881066 (+ 480us) server_negotiation.cc:299] Negotiation successful
0927 22:57:15.881225 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11533,"thread_start_us":11392,"threads_started":1}
I20260927 22:57:15.881729 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.860818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37123 (local address 127.4.74.130:36263)
0927 22:57:15.873109 (+ 12291us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:15.873138 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:15.873435 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:15.873959 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:15.873965 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:15.874008 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:15.874535 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.874544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.876078 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.876084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.878784 (+ 2700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.878805 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.880594 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.880602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.880802 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.880844 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 22:57:15.881021 (+ 177us) client_negotiation.cc:240] Negotiation successful
0927 22:57:15.881217 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12126,"mutex_wait_us":211,"spinlock_wait_cycles":48768,"thread_start_us":12170,"threads_started":1}
I20260927 22:57:15.881860 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.860834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:40491 (local address 127.4.74.130:60207)
0927 22:57:15.873237 (+ 12403us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:15.873254 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:15.873435 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:15.873959 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:15.873965 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:15.874005 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:15.874468 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.874483 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.876078 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.876084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.878784 (+ 2700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.878805 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.881005 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.881012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.881199 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.881227 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 22:57:15.881312 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 22:57:15.881479 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12289,"mutex_wait_us":240,"spinlock_wait_cycles":13952,"thread_start_us":11922,"threads_started":1}
I20260927 22:57:15.882570 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.860977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:60207 (local address 127.4.74.131:40491)
0927 22:57:15.872584 (+ 11607us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:15.872594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:15.873392 (+ 798us) server_negotiation.cc:407] Connection header received
0927 22:57:15.873624 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:15.873629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:15.873720 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:15.873871 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:15.874707 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.875912 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.879013 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.880311 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.880506 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.880586 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:15.882080 (+ 1494us) server_negotiation.cc:299] Negotiation successful
0927 22:57:15.882277 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11380,"thread_start_us":11272,"threads_started":1}
I20260927 22:57:15.883205 5679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2fe6c658554ae499906fea826f8372" candidate_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" is_pre_election: true
I20260927 22:57:15.883724 5679 raft_consensus.cc:2454] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4dbbf5fdef4b07a979f2b17f0c808e in term 0.
I20260927 22:57:15.884529 5959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2fe6c658554ae499906fea826f8372" candidate_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991e8dc6f4d448748f552ae87e9cc858" is_pre_election: true
I20260927 22:57:15.884605 5749 leader_election.cc:302] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4dbbf5fdef4b07a979f2b17f0c808e, e00ea55ce1714046a8a23fd8cd77a8f7; no voters:
I20260927 22:57:15.884837 5959 raft_consensus.cc:2454] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4dbbf5fdef4b07a979f2b17f0c808e in term 0.
I20260927 22:57:15.885324 6034 raft_consensus.cc:2781] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:15.885510 6034 raft_consensus.cc:482] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:15.885703 6034 raft_consensus.cc:3037] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:15.888474 6034 raft_consensus.cc:504] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.889267 6034 leader_election.cc:288] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [CANDIDATE]: Term 1 election: Requested vote from peers e00ea55ce1714046a8a23fd8cd77a8f7 (127.4.74.129:37123), 991e8dc6f4d448748f552ae87e9cc858 (127.4.74.131:40491)
I20260927 22:57:15.890048 5959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2fe6c658554ae499906fea826f8372" candidate_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991e8dc6f4d448748f552ae87e9cc858"
I20260927 22:57:15.890065 5679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2fe6c658554ae499906fea826f8372" candidate_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7"
I20260927 22:57:15.890388 5959 raft_consensus.cc:3037] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:15.890380 5679 raft_consensus.cc:3037] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:15.893814 5679 raft_consensus.cc:2454] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4dbbf5fdef4b07a979f2b17f0c808e in term 1.
I20260927 22:57:15.893797 5959 raft_consensus.cc:2454] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4dbbf5fdef4b07a979f2b17f0c808e in term 1.
I20260927 22:57:15.894557 5750 leader_election.cc:302] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 991e8dc6f4d448748f552ae87e9cc858, cc4dbbf5fdef4b07a979f2b17f0c808e, e00ea55ce1714046a8a23fd8cd77a8f7; no voters:
I20260927 22:57:15.895161 6034 raft_consensus.cc:2781] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:15.895879 6034 raft_consensus.cc:686] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [term 1 LEADER]: Becoming Leader. State: Replica: cc4dbbf5fdef4b07a979f2b17f0c808e, State: Running, Role: LEADER
I20260927 22:57:15.896828 6034 consensus_queue.cc:237] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } }
I20260927 22:57:15.897746 6047 consensus_queue.cc:1035] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:15.900815 6073 tablet_replica.cc:439] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: TxnStatusTablet state changed. Reason: New leader cc4dbbf5fdef4b07a979f2b17f0c808e. Latest consensus state: current_term: 1 leader_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } }
I20260927 22:57:15.901300 6073 tablet_replica.cc:442] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: This TxnStatusTablet replica's current role is: LEADER
I20260927 22:57:15.903252 5521 catalog_manager.cc:5495] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e reported cstate change: term changed from 0 to 1, leader changed from <none> to cc4dbbf5fdef4b07a979f2b17f0c808e (127.4.74.130). New cstate: current_term: 1 leader_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } health_report { overall_health: UNKNOWN } } }
I20260927 22:57:15.912410 6077 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 22:57:15.912753 6077 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 22:57:15.919915 6047 consensus_queue.cc:1035] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:15.972606 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.963902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49840 (local address 127.4.74.130:32837)
0927 22:57:15.964686 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:15.964694 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:15.964810 (+ 116us) server_negotiation.cc:407] Connection header received
0927 22:57:15.965281 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:15.965286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:15.965380 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:15.965568 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:15.966136 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.967135 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.968729 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.969286 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.969466 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.970363 (+ 897us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:15.971375 (+ 1012us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:15.971639 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:15.972032 (+ 393us) server_negotiation.cc:299] Negotiation successful
0927 22:57:15.972145 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":342,"threads_started":1}
I20260927 22:57:15.972596 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:15.964375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:32837 (local address 127.0.0.1:49840)
0927 22:57:15.964685 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:15.964710 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:15.964971 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:15.965608 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:15.965617 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:15.965636 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 22:57:15.965925 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.965937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.967295 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.967303 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.968547 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:15.968559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:15.969645 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:15.969654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:15.969956 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:15.970062 (+ 106us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 22:57:15.971672 (+ 1610us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 22:57:15.971749 (+ 77us) client_negotiation.cc:769] Sending connection context
0927 22:57:15.971900 (+ 151us) client_negotiation.cc:240] Negotiation successful
0927 22:57:15.972139 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260927 22:57:15.978586 5959 raft_consensus.cc:1261] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858 [term 1 FOLLOWER]: Refusing update from remote peer cc4dbbf5fdef4b07a979f2b17f0c808e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:15.978992 5679 raft_consensus.cc:1261] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 FOLLOWER]: Refusing update from remote peer cc4dbbf5fdef4b07a979f2b17f0c808e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:15.979485 6073 consensus_queue.cc:1035] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [LEADER]: Connected to new peer: Peer: permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:15.980150 6034 consensus_queue.cc:1035] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:15.990868 6039 tablet_replica.cc:439] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: TxnStatusTablet state changed. Reason: New leader cc4dbbf5fdef4b07a979f2b17f0c808e. Latest consensus state: current_term: 1 leader_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } }
I20260927 22:57:15.991258 6039 tablet_replica.cc:442] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:15.994292 6079 mvcc.cc:205] Tried to move back new op lower bound from 7334092128158285824 to 7334092127838814208. Current Snapshot: MvccSnapshot[applied={T|T < 7334092128158285824}]
I20260927 22:57:15.994675 6035 tablet_replica.cc:439] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: TxnStatusTablet state changed. Reason: New leader cc4dbbf5fdef4b07a979f2b17f0c808e. Latest consensus state: current_term: 1 leader_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } }
I20260927 22:57:15.995029 6035 tablet_replica.cc:442] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:15.996903 6075 tablet_replica.cc:439] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } }
I20260927 22:57:15.997610 6075 tablet_replica.cc:442] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: This TxnStatusTablet replica's current role is: LEADER
I20260927 22:57:15.997974 6075 tablet_replica.cc:439] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } }
I20260927 22:57:15.998209 6075 tablet_replica.cc:442] T da2fe6c658554ae499906fea826f8372 P cc4dbbf5fdef4b07a979f2b17f0c808e: This TxnStatusTablet replica's current role is: LEADER
I20260927 22:57:16.000841 6039 tablet_replica.cc:439] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } }
I20260927 22:57:16.001155 6039 tablet_replica.cc:442] T da2fe6c658554ae499906fea826f8372 P e00ea55ce1714046a8a23fd8cd77a8f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:16.002758 6083 mvcc.cc:205] Tried to move back new op lower bound from 7334092128158285824 to 7334092127838814208. Current Snapshot: MvccSnapshot[applied={T|T < 7334092128158285824}]
I20260927 22:57:16.005406 6078 mvcc.cc:205] Tried to move back new op lower bound from 7334092128158285824 to 7334092127838814208. Current Snapshot: MvccSnapshot[applied={T|T < 7334092128158285824 or (T in {7334092128158285824})}]
I20260927 22:57:16.009665 6035 tablet_replica.cc:439] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e00ea55ce1714046a8a23fd8cd77a8f7" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37123 } } peers { permanent_uuid: "cc4dbbf5fdef4b07a979f2b17f0c808e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 32837 } } peers { permanent_uuid: "991e8dc6f4d448748f552ae87e9cc858" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 40491 } } }
I20260927 22:57:16.010224 6035 tablet_replica.cc:442] T da2fe6c658554ae499906fea826f8372 P 991e8dc6f4d448748f552ae87e9cc858: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:16.034121 6093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:16.026733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:32837 (local address 127.4.74.129:54691)
0927 22:57:16.027626 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:16.027650 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:16.027898 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:16.028394 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:16.028398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:16.028409 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 22:57:16.028789 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:16.028798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.030392 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.030397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:16.031730 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:16.031746 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.032819 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.032824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:16.032968 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:16.033041 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 22:57:16.033516 (+ 475us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 22:57:16.033527 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 22:57:16.033603 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 22:57:16.033753 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":483,"threads_started":1}
I20260927 22:57:16.034102 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:16.026991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:54691 (local address 127.4.74.130:32837)
0927 22:57:16.027181 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:16.027190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:16.027782 (+ 592us) server_negotiation.cc:407] Connection header received
0927 22:57:16.028107 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:16.028117 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:16.028229 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:16.028356 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:16.028957 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.030176 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.031897 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.032575 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.032744 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:16.033253 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:16.033405 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:16.033503 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:16.033663 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 22:57:16.033761 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 22:57:16.081631 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:16.074110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37123 (local address 127.4.74.130:48149)
0927 22:57:16.074913 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:16.074963 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:16.075145 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:16.075650 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:16.075657 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:16.075675 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 22:57:16.076105 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:16.076118 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.077394 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.077402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:16.078961 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:16.078976 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.080052 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.080057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:16.080210 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:16.080283 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 22:57:16.080746 (+ 463us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 22:57:16.080761 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 22:57:16.080896 (+ 135us) client_negotiation.cc:240] Negotiation successful
0927 22:57:16.081084 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":413,"threads_started":1}
I20260927 22:57:16.081773 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:16.074138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:48149 (local address 127.4.74.129:37123)
0927 22:57:16.074342 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:16.074347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:16.075087 (+ 740us) server_negotiation.cc:407] Connection header received
0927 22:57:16.075329 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:16.075338 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:16.075451 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:16.075581 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:16.076295 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.077256 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.079117 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.079764 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.079901 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:16.080524 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:16.080646 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:16.080746 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:16.081154 (+ 408us) server_negotiation.cc:299] Negotiation successful
0927 22:57:16.081253 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 22:57:16.127453 5521 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:16.127843 5521 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:16.131390 5521 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a601c50393e04f46bef9a0364f2792ec: Sending DeleteTablet for 3 replicas of tablet fc2a37c6883d44ada112a57f8f29acbf
I20260927 22:57:16.132476 5658 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2a37c6883d44ada112a57f8f29acbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:16 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:59552
I20260927 22:57:16.132620 5799 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2a37c6883d44ada112a57f8f29acbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:16 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:37850
I20260927 22:57:16.134747 5939 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2a37c6883d44ada112a57f8f29acbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:16 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:49682
I20260927 22:57:16.152760 6098 tablet_replica.cc:330] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7: stopping tablet replica
I20260927 22:57:16.153023 6098 raft_consensus.cc:2229] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:57:16.153591 6101 tablet_replica.cc:330] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858: stopping tablet replica
I20260927 22:57:16.153630 6098 raft_consensus.cc:2258] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:16.153893 6101 raft_consensus.cc:2229] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:16.154280 6101 raft_consensus.cc:2258] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:16.154860 6098 ts_tablet_manager.cc:1905] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:16.156431 6101 ts_tablet_manager.cc:1905] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:16.166998 6098 ts_tablet_manager.cc:1918] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:57:16.167233 6098 log.cc:1196] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/fc2a37c6883d44ada112a57f8f29acbf
I20260927 22:57:16.167739 6098 ts_tablet_manager.cc:1939] T fc2a37c6883d44ada112a57f8f29acbf P e00ea55ce1714046a8a23fd8cd77a8f7: Deleting consensus metadata
I20260927 22:57:16.169014 5506 catalog_manager.cc:4841] TS e00ea55ce1714046a8a23fd8cd77a8f7 (127.4.74.129:37123): tablet fc2a37c6883d44ada112a57f8f29acbf (table test-table [id=6e2998a6d9c84483af3f6e3e5877dbe4]) successfully deleted
I20260927 22:57:16.179473 6099 tablet_replica.cc:330] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e: stopping tablet replica
I20260927 22:57:16.179778 6099 raft_consensus.cc:2229] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:16.180094 6099 raft_consensus.cc:2258] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:16.181282 6101 ts_tablet_manager.cc:1918] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:57:16.181435 6099 ts_tablet_manager.cc:1905] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:16.181488 6101 log.cc:1196] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/fc2a37c6883d44ada112a57f8f29acbf
I20260927 22:57:16.182829 6101 ts_tablet_manager.cc:1939] T fc2a37c6883d44ada112a57f8f29acbf P 991e8dc6f4d448748f552ae87e9cc858: Deleting consensus metadata
I20260927 22:57:16.184443 5507 catalog_manager.cc:4841] TS 991e8dc6f4d448748f552ae87e9cc858 (127.4.74.131:40491): tablet fc2a37c6883d44ada112a57f8f29acbf (table test-table [id=6e2998a6d9c84483af3f6e3e5877dbe4]) successfully deleted
Sep 27 22:57:16 dist-test-slave-5pd3 krb5kdc[5477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549834, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:16.195182 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:16.161037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55590 (local address 127.4.74.129:37123)
0927 22:57:16.161255 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:16.161262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:16.167138 (+ 5876us) server_negotiation.cc:407] Connection header received
0927 22:57:16.167838 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:16.167846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:16.167974 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:16.168128 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:16.175780 (+ 7652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.177418 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.179930 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.180616 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.180779 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:16.187521 (+ 6742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:16.187588 (+ 67us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:16.187624 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:16.187674 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:16.191254 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:16.192339 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:16.192357 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:16.192360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:16.192538 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:16.192965 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:16.192971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:16.192973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:16.193462 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:16.193817 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:16.194253 (+ 436us) server_negotiation.cc:299] Negotiation successful
0927 22:57:16.194538 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260927 22:57:16.196087 5724 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55590
I20260927 22:57:16.197417 6099 ts_tablet_manager.cc:1918] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:57:16.197587 6099 log.cc:1196] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/fc2a37c6883d44ada112a57f8f29acbf
I20260927 22:57:16.198058 6099 ts_tablet_manager.cc:1939] T fc2a37c6883d44ada112a57f8f29acbf P cc4dbbf5fdef4b07a979f2b17f0c808e: Deleting consensus metadata
I20260927 22:57:16.199348 5508 catalog_manager.cc:4841] TS cc4dbbf5fdef4b07a979f2b17f0c808e (127.4.74.130:32837): tablet fc2a37c6883d44ada112a57f8f29acbf (table test-table [id=6e2998a6d9c84483af3f6e3e5877dbe4]) successfully deleted
I20260927 22:57:16.219360 6105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:16.200465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55810 (local address 127.4.74.190:39797)
0927 22:57:16.204122 (+ 3657us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:16.204128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:16.204150 (+ 22us) server_negotiation.cc:407] Connection header received
0927 22:57:16.204221 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:16.204225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:16.204305 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:16.204484 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:16.206273 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.207191 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.208937 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.209433 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.209602 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:16.212644 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:16.212649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:16.212654 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:16.212694 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:16.216064 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:16.217005 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:16.217011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:16.217015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:16.217211 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:16.217630 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:16.217636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:16.217640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:16.218038 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:16.218335 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:16.218678 (+ 343us) server_negotiation.cc:299] Negotiation successful
0927 22:57:16.218960 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3506,"thread_start_us":3307,"threads_started":1}
W20260927 22:57:16.220350 5521 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55810
I20260927 22:57:16.222728 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 5593
I20260927 22:57:16.238484 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:16.223744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55594 (local address 127.4.74.129:37123)
0927 22:57:16.224139 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:16.224145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:16.224414 (+ 269us) server_negotiation.cc:407] Connection header received
0927 22:57:16.224692 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:16.224699 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:16.224787 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:16.224924 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:16.226172 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.226942 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.228322 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.228771 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.228926 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:16.232461 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:16.232466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:16.232470 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:16.232508 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:16.235340 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:16.236314 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:16.236319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:16.236321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:16.236472 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:16.236951 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:16.236957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:16.236960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:16.237306 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:16.237529 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:16.237937 (+ 408us) server_negotiation.cc:299] Negotiation successful
0927 22:57:16.238139 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
W20260927 22:57:16.239144 5724 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55594
W20260927 22:57:16.239871 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 5593: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 22:57:16.240101 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 5593
I20260927 22:57:16.265326 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 5733
Sep 27 22:57:16 dist-test-slave-5pd3 krb5kdc[5477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549834, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:57:16.285606 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:16.266437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49844 (local address 127.4.74.130:32837)
0927 22:57:16.266682 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:16.266687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:16.266978 (+ 291us) server_negotiation.cc:407] Connection header received
0927 22:57:16.267212 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:16.267217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:16.267331 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:16.267452 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:16.269155 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.270179 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.271749 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.272278 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.272437 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:16.277715 (+ 5278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:16.277735 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:16.277757 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:16.277813 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:16.282084 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:16.283080 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:16.283088 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:16.283091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:16.283245 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:16.283697 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:16.283703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:16.283705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:16.284164 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:16.284551 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:16.284862 (+ 311us) server_negotiation.cc:299] Negotiation successful
0927 22:57:16.285153 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260927 22:57:16.286589 5864 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49844
W20260927 22:57:16.287541 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 5733: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 22:57:16.287779 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 5733
I20260927 22:57:16.309966 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 5873
Sep 27 22:57:16 dist-test-slave-5pd3 krb5kdc[5477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549834, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:57:16.326620 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:16.310893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33894 (local address 127.4.74.131:40491)
0927 22:57:16.311148 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:16.311152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:16.311175 (+ 23us) server_negotiation.cc:407] Connection header received
0927 22:57:16.311242 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:16.311246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:16.311320 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:16.311434 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:16.312867 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.313555 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.314922 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:16.315350 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:16.315457 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:16.320439 (+ 4982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:16.320456 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:16.320473 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:16.320511 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:16.323487 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:16.324591 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:16.324598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:16.324601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:16.324750 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:16.325205 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:16.325208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:16.325210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:16.325573 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:16.325816 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:16.326136 (+ 320us) server_negotiation.cc:299] Negotiation successful
0927 22:57:16.326340 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260927 22:57:16.327705 6004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33894
W20260927 22:57:16.328565 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 5873: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 22:57:16.328766 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 5873
I20260927 22:57:16.347215 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 5490
I20260927 22:57:16.348709 5562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:16.465432 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 5490
2026-09-27T22:57:16Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5518 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:16 dist-test-slave-5pd3 krb5kdc[6110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:16 dist-test-slave-5pd3 krb5kdc[6110](info): set up 2 sockets
Sep 27 22:57:16 dist-test-slave-5pd3 krb5kdc[6110](info): commencing operation
krb5kdc: starting...
W20260927 22:57:17.583869 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:17 dist-test-slave-5pd3 krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:57:17.799820 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:39421
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:39421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:18.207993 6123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:18.208668 6123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:18.208891 6123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:18.217696 6123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:18.217856 6123 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 22:57:18.217902 6123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:18.217960 6123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:18.218021 6123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:18.218091 6123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:18.228982 6123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:39421
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:39421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:18.230398 6123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:18.231838 6123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:18.246338 6131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:18.246490 6128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:18.246486 6129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:18.247895 6123 server_base.cc:1034] running on GCE node
I20260927 22:57:18.249003 6123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:18.251083 6123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:18.252416 6123 hybrid_clock.cc:648] HybridClock initialized: now 1790549838252349 us; error 50 us; skew 500 ppm
I20260927 22:57:18.252817 6123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:18 dist-test-slave-5pd3 krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549838, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:18.257082 6123 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:57:18.259294 6123 webserver.cc:460] Webserver started at http://127.4.74.190:44737/ using document root <none> and password file <none>
I20260927 22:57:18.260353 6123 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:18.260509 6123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:18.260903 6123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:18.264884 6123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "9dfff65842c34f488ff1623336e3eb5c"
format_stamp: "Formatted at 2026-09-27 22:57:18 on dist-test-slave-5pd3"
server_key: "66e917d3c881c815b60edd6edefc324d"
server_key_iv: "d37fdd424fa7141318eca0dbf1528f8a"
server_key_version: "encryptionkey@0"
I20260927 22:57:18.265858 6123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "9dfff65842c34f488ff1623336e3eb5c"
format_stamp: "Formatted at 2026-09-27 22:57:18 on dist-test-slave-5pd3"
server_key: "66e917d3c881c815b60edd6edefc324d"
server_key_iv: "d37fdd424fa7141318eca0dbf1528f8a"
server_key_version: "encryptionkey@0"
I20260927 22:57:18.273182 6123 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260927 22:57:18.277902 6138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:18.279803 6123 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 22:57:18.280053 6123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "9dfff65842c34f488ff1623336e3eb5c"
format_stamp: "Formatted at 2026-09-27 22:57:18 on dist-test-slave-5pd3"
server_key: "66e917d3c881c815b60edd6edefc324d"
server_key_iv: "d37fdd424fa7141318eca0dbf1528f8a"
server_key_version: "encryptionkey@0"
I20260927 22:57:18.280345 6123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:18.303968 6123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:18.309546 6123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:18.309984 6123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:18.337950 6123 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:39421
I20260927 22:57:18.337990 6199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:39421 every 8 connection(s)
I20260927 22:57:18.339839 6123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:18.343782 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 6123
I20260927 22:57:18.344026 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:18.344884 4394 external_mini_cluster.cc:1428] Setting key 4cc33df9e2abe23f9c24f744f4d61867
I20260927 22:57:18.345247 6200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:18.361456 6200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c: Bootstrap starting.
Sep 27 22:57:18 dist-test-slave-5pd3 krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:18.365330 6200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:18.366606 6200 log.cc:824] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:18.369767 6200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c: No bootstrap required, opened a new log
I20260927 22:57:18.373031 6203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:18.347412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.4.74.190:39421)
0927 22:57:18.348461 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:18.348479 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:18.348537 (+ 58us) server_negotiation.cc:407] Connection header received
0927 22:57:18.350633 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:18.350662 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:18.351178 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:18.351692 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:18.353568 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:18.355119 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:18.357236 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:18.358222 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:18.358439 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:18.363821 (+ 5382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:18.363835 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:18.363857 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:18.363911 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:18.367901 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:18.368840 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:18.368848 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:18.368856 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:18.369072 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:18.369497 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:18.369500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:18.369503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:18.370210 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:18.370612 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:18.371029 (+ 417us) server_negotiation.cc:299] Negotiation successful
0927 22:57:18.371625 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":784,"thread_start_us":453,"threads_started":1}
I20260927 22:57:18.378187 6200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfff65842c34f488ff1623336e3eb5c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39421 } }
I20260927 22:57:18.378604 6200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:18.378718 6200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dfff65842c34f488ff1623336e3eb5c, State: Initialized, Role: FOLLOWER
I20260927 22:57:18.379376 6200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfff65842c34f488ff1623336e3eb5c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39421 } }
I20260927 22:57:18.379678 6200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:18.379829 6200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:18.380041 6200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:18.382582 6200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfff65842c34f488ff1623336e3eb5c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39421 } }
I20260927 22:57:18.383088 6200 leader_election.cc:302] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dfff65842c34f488ff1623336e3eb5c; no voters:
I20260927 22:57:18.383776 6200 leader_election.cc:288] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:18.383952 6205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:18.385751 6205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [term 1 LEADER]: Becoming Leader. State: Replica: 9dfff65842c34f488ff1623336e3eb5c, State: Running, Role: LEADER
I20260927 22:57:18.386693 6205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfff65842c34f488ff1623336e3eb5c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39421 } }
I20260927 22:57:18.387018 6200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:18.390364 6207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dfff65842c34f488ff1623336e3eb5c. Latest consensus state: current_term: 1 leader_uuid: "9dfff65842c34f488ff1623336e3eb5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfff65842c34f488ff1623336e3eb5c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39421 } } }
I20260927 22:57:18.390916 6207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:18.397504 6212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:18.397274 6206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9dfff65842c34f488ff1623336e3eb5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfff65842c34f488ff1623336e3eb5c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39421 } } }
I20260927 22:57:18.397742 6206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:18.404393 6212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:18.415933 6212 catalog_manager.cc:1347] Generated new cluster ID: 71fb60ba682e4a8e93db1695e4bffd3d
I20260927 22:57:18.416116 6212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:18.432505 6212 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:18.434598 6212 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:18.446559 6212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c: Generated new TSK 0
I20260927 22:57:18.447791 6212 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:57:18.515720 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39421
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:18.939760 6226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:18.940171 6226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:18.940361 6226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:18.949254 6226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:18.949440 6226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:18.949592 6226 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 22:57:18.949698 6226 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 22:57:18.949802 6226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:57:18.961602 6226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.74.190:39421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:18.963351 6226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:18.965135 6226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:18.981907 6232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:18.982080 6231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:18.982059 6234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:18.983440 6226 server_base.cc:1034] running on GCE node
I20260927 22:57:18.984292 6226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:18.985546 6226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:18.986838 6226 hybrid_clock.cc:648] HybridClock initialized: now 1790549838986773 us; error 61 us; skew 500 ppm
I20260927 22:57:18.987259 6226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:18 dist-test-slave-5pd3 krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549838, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:18.991079 6226 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:57:18.993021 6226 webserver.cc:460] Webserver started at http://127.4.74.129:46223/ using document root <none> and password file <none>
I20260927 22:57:18.993865 6226 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:18.994032 6226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:18.994383 6226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:18.997906 6226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "513b026fe15b45ca944708db85567088"
format_stamp: "Formatted at 2026-09-27 22:57:18 on dist-test-slave-5pd3"
server_key: "91b3122e1180b36cb504fd560c113b33"
server_key_iv: "1538f4df2c3a82789d8249ea7454284e"
server_key_version: "encryptionkey@0"
I20260927 22:57:18.998762 6226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "513b026fe15b45ca944708db85567088"
format_stamp: "Formatted at 2026-09-27 22:57:18 on dist-test-slave-5pd3"
server_key: "91b3122e1180b36cb504fd560c113b33"
server_key_iv: "1538f4df2c3a82789d8249ea7454284e"
server_key_version: "encryptionkey@0"
I20260927 22:57:19.005427 6226 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260927 22:57:19.010016 6241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:19.012336 6226 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 22:57:19.012624 6226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "513b026fe15b45ca944708db85567088"
format_stamp: "Formatted at 2026-09-27 22:57:18 on dist-test-slave-5pd3"
server_key: "91b3122e1180b36cb504fd560c113b33"
server_key_iv: "1538f4df2c3a82789d8249ea7454284e"
server_key_version: "encryptionkey@0"
I20260927 22:57:19.012905 6226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:19.027906 6226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:19.033272 6226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:19.033619 6226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:19.037132 6226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:19.037284 6226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:19.037456 6226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:19.037578 6226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 22:57:19 dist-test-slave-5pd3 krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549838, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:19.068424 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:19.039531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:35011 (local address 127.4.74.190:39421)
0927 22:57:19.040184 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:19.040192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:19.040432 (+ 240us) server_negotiation.cc:407] Connection header received
0927 22:57:19.043435 (+ 3003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:19.043441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:19.043532 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:19.043718 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:19.047245 (+ 3527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.048548 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.052106 (+ 3558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.052592 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.052764 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:19.058620 (+ 5856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:19.058628 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:19.058633 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:19.058685 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:19.062965 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:19.064762 (+ 1797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:19.064772 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:19.064779 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:19.065011 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:19.065864 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:19.065871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:19.065875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:19.066387 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:19.066578 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:19.067696 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0927 22:57:19.067995 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":886,"threads_started":1}
I20260927 22:57:19.069788 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:19.038829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39421 (local address 127.4.74.129:35011)
0927 22:57:19.040196 (+ 1367us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:19.040256 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:19.042920 (+ 2664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:19.044447 (+ 1527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:19.044466 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:19.045642 (+ 1176us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:19.046984 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:19.047011 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.048940 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.048948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:19.051806 (+ 2858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:19.051839 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.052897 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.052907 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:19.053201 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:19.054179 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:19.054206 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:19.058140 (+ 3934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:19.063317 (+ 5177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:19.063336 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:19.063383 (+ 47us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:19.064473 (+ 1090us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:19.065210 (+ 737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:19.065219 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:19.065225 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:19.065658 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:19.066643 (+ 985us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:19.066654 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:19.067166 (+ 512us) client_negotiation.cc:769] Sending connection context
0927 22:57:19.067523 (+ 357us) client_negotiation.cc:240] Negotiation successful
0927 22:57:19.068260 (+ 737us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"spinlock_wait_cycles":19840,"thread_start_us":770,"threads_started":1}
I20260927 22:57:19.098259 6226 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:34697
I20260927 22:57:19.098387 6361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:34697 every 8 connection(s)
I20260927 22:57:19.100471 6226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:57:19.100775 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:19.079591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:59315 (local address 127.4.74.190:39421)
0927 22:57:19.079851 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:19.079860 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:19.080911 (+ 1051us) server_negotiation.cc:407] Connection header received
0927 22:57:19.081143 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:19.081153 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:19.081277 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:19.081452 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:19.083773 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.085025 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.087928 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.088715 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.088902 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:19.093135 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:19.093146 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:19.093154 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:19.093208 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:19.096958 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:19.098298 (+ 1340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:19.098305 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:19.098312 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:19.098478 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:19.099032 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:19.099036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:19.099038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:19.099359 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:19.099493 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:19.100220 (+ 727us) server_negotiation.cc:299] Negotiation successful
0927 22:57:19.100437 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260927 22:57:19.100996 6322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:19.079812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39421 (local address 127.4.74.129:59315)
0927 22:57:19.080759 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:19.080782 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:19.080974 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:19.081711 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:19.081720 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:19.082701 (+ 981us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:19.083456 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:19.083479 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.085241 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.085248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:19.087702 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:19.087722 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.089051 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.089064 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:19.089365 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:19.090412 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:19.090447 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:19.092705 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:19.097173 (+ 4468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:19.097213 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:19.097221 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:19.098086 (+ 865us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:19.098674 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:19.098680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:19.098684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:19.098856 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:19.099501 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:19.099506 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:19.099807 (+ 301us) client_negotiation.cc:769] Sending connection context
0927 22:57:19.100036 (+ 229us) client_negotiation.cc:240] Negotiation successful
0927 22:57:19.100512 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":629,"threads_started":1}
I20260927 22:57:19.102216 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 6226
I20260927 22:57:19.102489 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:57:19.103238 4394 external_mini_cluster.cc:1428] Setting key bb9938043baa99469f2ed77c263b1119
I20260927 22:57:19.104993 6362 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39421
I20260927 22:57:19.105306 6362 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:19.106328 6362 heartbeater.cc:511] Master 127.4.74.190:39421 requested a full tablet report, sending...
I20260927 22:57:19.109236 6154 ts_manager.cc:194] Registered new tserver with Master: 513b026fe15b45ca944708db85567088 (127.4.74.129:34697)
I20260927 22:57:19.111372 6154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:35011
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:57:19.150748 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39421
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:19.505736 6366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:19.506106 6366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:19.506294 6366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:19.514218 6366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:19.514385 6366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:19.514547 6366 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 22:57:19.514638 6366 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 22:57:19.514748 6366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:57:19.525074 6366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.74.190:39421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:19.526490 6366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:19.528151 6366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:19.543375 6371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:19.543301 6374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:19.543251 6372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:19.544467 6366 server_base.cc:1034] running on GCE node
I20260927 22:57:19.545230 6366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:19.546279 6366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:19.547662 6366 hybrid_clock.cc:648] HybridClock initialized: now 1790549839547528 us; error 54 us; skew 500 ppm
I20260927 22:57:19.548060 6366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:19 dist-test-slave-5pd3 krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549839, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:19.551752 6366 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:57:19.553439 6366 webserver.cc:460] Webserver started at http://127.4.74.130:42093/ using document root <none> and password file <none>
I20260927 22:57:19.554172 6366 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:19.554324 6366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:19.554647 6366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:19.557617 6366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "2313a7f01817402fbd824734bc8608fe"
format_stamp: "Formatted at 2026-09-27 22:57:19 on dist-test-slave-5pd3"
server_key: "de1b8d152fedad90d4223abb38c78cbf"
server_key_iv: "c2614b93feec1c68a1e8bf7f454175a0"
server_key_version: "encryptionkey@0"
I20260927 22:57:19.558405 6366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "2313a7f01817402fbd824734bc8608fe"
format_stamp: "Formatted at 2026-09-27 22:57:19 on dist-test-slave-5pd3"
server_key: "de1b8d152fedad90d4223abb38c78cbf"
server_key_iv: "c2614b93feec1c68a1e8bf7f454175a0"
server_key_version: "encryptionkey@0"
I20260927 22:57:19.564059 6366 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260927 22:57:19.568341 6381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:19.570695 6366 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:57:19.570971 6366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "2313a7f01817402fbd824734bc8608fe"
format_stamp: "Formatted at 2026-09-27 22:57:19 on dist-test-slave-5pd3"
server_key: "de1b8d152fedad90d4223abb38c78cbf"
server_key_iv: "c2614b93feec1c68a1e8bf7f454175a0"
server_key_version: "encryptionkey@0"
I20260927 22:57:19.571228 6366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:19.592355 6366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:19.596907 6366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:19.597237 6366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:19.600450 6366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:19.600620 6366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:19.600793 6366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:19.600898 6366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 22:57:19 dist-test-slave-5pd3 krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549839, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:19.631457 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:19.601578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:37905 (local address 127.4.74.190:39421)
0927 22:57:19.602148 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:19.602155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:19.603769 (+ 1614us) server_negotiation.cc:407] Connection header received
0927 22:57:19.606279 (+ 2510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:19.606287 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:19.606410 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:19.606539 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:19.609737 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.610965 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.613718 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.614327 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.614487 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:19.622226 (+ 7739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:19.622236 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:19.622244 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:19.622306 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:19.626683 (+ 4377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:19.628270 (+ 1587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:19.628280 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:19.628287 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:19.628507 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:19.629160 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:19.629169 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:19.629174 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:19.629622 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:19.629814 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:19.630711 (+ 897us) server_negotiation.cc:299] Negotiation successful
0927 22:57:19.631013 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":438,"threads_started":1}
I20260927 22:57:19.633483 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:19.602265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39421 (local address 127.4.74.130:37905)
0927 22:57:19.603472 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:19.603579 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:19.605865 (+ 2286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:19.606925 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:19.606944 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:19.608196 (+ 1252us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:19.609431 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:19.609470 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.611174 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.611180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:19.613428 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:19.613448 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.614686 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.614697 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:19.615211 (+ 514us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:19.616572 (+ 1361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:19.616641 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:19.621738 (+ 5097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:19.626972 (+ 5234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:19.626985 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:19.627014 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:19.628003 (+ 989us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:19.628698 (+ 695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:19.628704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:19.628709 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:19.629013 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:19.629772 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:19.629783 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:19.630288 (+ 505us) client_negotiation.cc:769] Sending connection context
0927 22:57:19.630615 (+ 327us) client_negotiation.cc:240] Negotiation successful
0927 22:57:19.631308 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":481,"threads_started":1}
I20260927 22:57:19.666422 6366 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:38619
I20260927 22:57:19.666543 6501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:38619 every 8 connection(s)
I20260927 22:57:19.667990 6366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:57:19.671082 6502 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39421
I20260927 22:57:19.671425 6502 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:19.671988 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:19.652947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39421 (local address 127.4.74.130:51233)
0927 22:57:19.653802 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:19.653833 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:19.654051 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:19.654683 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:19.654691 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:19.655753 (+ 1062us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:19.656657 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:19.656677 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.658493 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.658499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:19.659684 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:19.659716 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.661080 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.661091 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:19.661389 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:19.662434 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:19.662471 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:19.664159 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:19.668264 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:19.668271 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:19.668276 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:19.668975 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:19.669625 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:19.669630 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:19.669633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:19.669873 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:19.670782 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:19.670787 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:19.671026 (+ 239us) client_negotiation.cc:769] Sending connection context
0927 22:57:19.671153 (+ 127us) client_negotiation.cc:240] Negotiation successful
0927 22:57:19.671592 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":460,"threads_started":1}
I20260927 22:57:19.672101 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:19.652881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:51233 (local address 127.4.74.190:39421)
0927 22:57:19.653088 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:19.653094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:19.653973 (+ 879us) server_negotiation.cc:407] Connection header received
0927 22:57:19.654335 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:19.654349 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:19.654468 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:19.654593 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:19.656953 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.658314 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.659924 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:19.660699 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:19.660870 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:19.664437 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:19.664443 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:19.664447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:19.664492 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:19.668014 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:19.669175 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:19.669180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:19.669184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:19.669331 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:19.670065 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:19.670072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:19.670077 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:19.670577 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:19.670739 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:19.671365 (+ 626us) server_negotiation.cc:299] Negotiation successful
0927 22:57:19.671684 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 22:57:19.672636 6502 heartbeater.cc:511] Master 127.4.74.190:39421 requested a full tablet report, sending...
I20260927 22:57:19.674592 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 6366
I20260927 22:57:19.674819 6154 ts_manager.cc:194] Registered new tserver with Master: 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619)
I20260927 22:57:19.674963 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:57:19.675704 4394 external_mini_cluster.cc:1428] Setting key f431a73f05c787bafe08109112eda695
I20260927 22:57:19.676205 6154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:37905
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:57:19.724129 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39421
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:20.080717 6506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:20.081013 6506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:20.081116 6506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:20.089170 6506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:20.089305 6506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:20.089393 6506 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 22:57:20.089450 6506 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 22:57:20.089514 6506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:57:20.109297 6506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.74.190:39421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:20.111291 6506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:20.113030 6506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 22:57:20.115319 6362 heartbeater.cc:503] Master 127.4.74.190:39421 was elected leader, sending a full tablet report...
W20260927 22:57:20.128361 6511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:20.128700 6512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:20.128758 6514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:20.129828 6506 server_base.cc:1034] running on GCE node
I20260927 22:57:20.130604 6506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:20.131829 6506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:20.133131 6506 hybrid_clock.cc:648] HybridClock initialized: now 1790549840133062 us; error 66 us; skew 500 ppm
I20260927 22:57:20.133534 6506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:20 dist-test-slave-5pd3 krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549840, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:20.137429 6506 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:57:20.139629 6506 webserver.cc:460] Webserver started at http://127.4.74.131:40031/ using document root <none> and password file <none>
I20260927 22:57:20.140575 6506 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:20.140700 6506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:20.141047 6506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:20.144820 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "3e819715daae426a9cc50cb06fef6b23"
format_stamp: "Formatted at 2026-09-27 22:57:20 on dist-test-slave-5pd3"
server_key: "311150937f35a90375d63ad84413f7b1"
server_key_iv: "ba9077957d3f22f1976ada9a94900a98"
server_key_version: "encryptionkey@0"
I20260927 22:57:20.145727 6506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "3e819715daae426a9cc50cb06fef6b23"
format_stamp: "Formatted at 2026-09-27 22:57:20 on dist-test-slave-5pd3"
server_key: "311150937f35a90375d63ad84413f7b1"
server_key_iv: "ba9077957d3f22f1976ada9a94900a98"
server_key_version: "encryptionkey@0"
I20260927 22:57:20.152734 6506 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260927 22:57:20.157572 6521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:20.159452 6506 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260927 22:57:20.159746 6506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "3e819715daae426a9cc50cb06fef6b23"
format_stamp: "Formatted at 2026-09-27 22:57:20 on dist-test-slave-5pd3"
server_key: "311150937f35a90375d63ad84413f7b1"
server_key_iv: "ba9077957d3f22f1976ada9a94900a98"
server_key_version: "encryptionkey@0"
I20260927 22:57:20.159994 6506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:20.195724 6506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:20.201623 6506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:20.201958 6506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:20.205955 6506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:20.206102 6506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:20.206311 6506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:20.206442 6506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 22:57:20 dist-test-slave-5pd3 krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549840, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:20.239516 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.206610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:47835 (local address 127.4.74.190:39421)
0927 22:57:20.207932 (+ 1322us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.207937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.208980 (+ 1043us) server_negotiation.cc:407] Connection header received
0927 22:57:20.211916 (+ 2936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.211927 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.212041 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.212216 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:20.215063 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.216528 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.220231 (+ 3703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.221104 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.221308 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.228425 (+ 7117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:20.228436 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:20.228441 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:20.228509 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:20.233364 (+ 4855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:20.234974 (+ 1610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:20.234984 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:20.234991 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:20.235202 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:20.236233 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:20.236238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:20.236241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:20.236700 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:20.236905 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.238719 (+ 1814us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.239021 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1190,"thread_start_us":1086,"threads_started":1}
I20260927 22:57:20.241667 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.207327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39421 (local address 127.4.74.131:47835)
0927 22:57:20.208678 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:20.208745 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:20.211386 (+ 2641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:20.212535 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:20.212546 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:20.213933 (+ 1387us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:20.214768 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.214791 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.216796 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.216806 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.219884 (+ 3078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.219931 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.221578 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.221596 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.222012 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.223075 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:20.223106 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:20.227994 (+ 4888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:20.233664 (+ 5670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:20.233675 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:20.233696 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:20.234739 (+ 1043us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:20.235388 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:20.235393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:20.235397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:20.235943 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:20.236881 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:20.236893 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:20.237994 (+ 1101us) client_negotiation.cc:769] Sending connection context
0927 22:57:20.238592 (+ 598us) client_negotiation.cc:240] Negotiation successful
0927 22:57:20.239394 (+ 802us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"thread_start_us":711,"threads_started":1}
I20260927 22:57:20.261552 6506 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:35309
I20260927 22:57:20.261622 6641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:35309 every 8 connection(s)
I20260927 22:57:20.263295 6506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:57:20.266628 6642 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39421
I20260927 22:57:20.266960 6642 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:20.267431 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 6506
I20260927 22:57:20.267649 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:57:20.268071 6642 heartbeater.cc:511] Master 127.4.74.190:39421 requested a full tablet report, sending...
I20260927 22:57:20.268260 4394 external_mini_cluster.cc:1428] Setting key 1b3b7ab9551f83295ffc10f26e39dd9b
I20260927 22:57:20.270001 6155 ts_manager.cc:194] Registered new tserver with Master: 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131:35309)
I20260927 22:57:20.270610 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.250822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:50665 (local address 127.4.74.190:39421)
0927 22:57:20.251819 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.251830 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.252194 (+ 364us) server_negotiation.cc:407] Connection header received
0927 22:57:20.252433 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.252441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.252542 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.252655 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:20.254116 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.255324 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.258860 (+ 3536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.259473 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.259721 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.263002 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:20.263009 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:20.263013 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:20.263061 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:20.266946 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:20.268051 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:20.268058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:20.268063 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:20.268278 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:20.268874 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:20.268879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:20.268882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:20.269296 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:20.269506 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.269926 (+ 420us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.270186 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787}
I20260927 22:57:20.270637 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.251128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39421 (local address 127.4.74.131:50665)
0927 22:57:20.252047 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:20.252067 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:20.252293 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:20.252887 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:20.252894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:20.253630 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:20.253956 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.253966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.255598 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.255615 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.258352 (+ 2737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.258508 (+ 156us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.259830 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.259840 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.260163 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.261364 (+ 1201us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:20.261396 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:20.262665 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:20.267178 (+ 4513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:20.267184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:20.267189 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:20.267868 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:20.268482 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:20.268490 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:20.268494 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:20.268719 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:20.269441 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:20.269446 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:20.269650 (+ 204us) client_negotiation.cc:769] Sending connection context
0927 22:57:20.269796 (+ 146us) client_negotiation.cc:240] Negotiation successful
0927 22:57:20.270176 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":698,"threads_started":1}
I20260927 22:57:20.271488 6155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:47835
I20260927 22:57:20.272969 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 22:57:20 dist-test-slave-5pd3 krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 22:57:20 dist-test-slave-5pd3 krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:20.332211 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.314500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.4.74.190:39421)
0927 22:57:20.314806 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.314816 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.315230 (+ 414us) server_negotiation.cc:407] Connection header received
0927 22:57:20.315506 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.315511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.315649 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.315769 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:20.317403 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.318185 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.319812 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.320503 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.320643 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.326014 (+ 5371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:20.326021 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:20.326025 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:20.326072 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:20.329077 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:20.329995 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:20.330000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:20.330002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:20.330167 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:20.330679 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:20.330683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:20.330685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:20.331036 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:20.331259 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.331710 (+ 451us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.331935 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260927 22:57:20.337910 6155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:57:20.349982 6155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:20.382910 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.365683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36346 (local address 127.4.74.129:34697)
0927 22:57:20.366832 (+ 1149us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.366850 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.369087 (+ 2237us) server_negotiation.cc:407] Connection header received
0927 22:57:20.369588 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.369598 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.369968 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.370209 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:20.371806 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.374010 (+ 2204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.378329 (+ 4319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.380611 (+ 2282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.380775 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.381105 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.382146 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.382424 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":819,"threads_started":1}
I20260927 22:57:20.384254 6655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.365484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:34697 (local address 127.0.0.1:36346)
0927 22:57:20.368896 (+ 3412us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:20.368961 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:20.369288 (+ 327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:20.370642 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:20.370655 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:20.370706 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:20.371487 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.371507 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.374303 (+ 2796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.374312 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.378023 (+ 3711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.378052 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.381377 (+ 3325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.381388 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.381725 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.381817 (+ 92us) client_negotiation.cc:769] Sending connection context
0927 22:57:20.383234 (+ 1417us) client_negotiation.cc:240] Negotiation successful
0927 22:57:20.383592 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2428,"mutex_wait_us":6,"thread_start_us":1606,"threads_started":1}
I20260927 22:57:20.385149 6657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.366589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:38619 (local address 127.0.0.1:45788)
0927 22:57:20.369268 (+ 2679us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:20.369292 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:20.369524 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:20.370839 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:20.370848 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:20.370899 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:20.371819 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.371840 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.374924 (+ 3084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.374947 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.379200 (+ 4253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.379233 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.382970 (+ 3737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.382981 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.383749 (+ 768us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.383822 (+ 73us) client_negotiation.cc:769] Sending connection context
0927 22:57:20.383951 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 22:57:20.384219 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2490,"mutex_wait_us":42,"thread_start_us":2429,"threads_started":1}
I20260927 22:57:20.386003 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.367918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45788 (local address 127.4.74.130:38619)
0927 22:57:20.369251 (+ 1333us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.369264 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.369432 (+ 168us) server_negotiation.cc:407] Connection header received
0927 22:57:20.369809 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.369820 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.370167 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.370459 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:20.372174 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.374588 (+ 2414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.379575 (+ 4987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.382098 (+ 2523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.382362 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.382722 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.385426 (+ 2704us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.385655 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":903,"threads_started":1}
I20260927 22:57:20.389158 6292 tablet_service.cc:1467] Processing CreateTablet for tablet d94ac5a8eb7541949ef388de245e5a39 (DEFAULT_TABLE table=test-table [id=15201661024347e9b2e6146b2b03ec99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:20.390743 6292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d94ac5a8eb7541949ef388de245e5a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:20.391390 6432 tablet_service.cc:1467] Processing CreateTablet for tablet d94ac5a8eb7541949ef388de245e5a39 (DEFAULT_TABLE table=test-table [id=15201661024347e9b2e6146b2b03ec99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:20.393496 6432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d94ac5a8eb7541949ef388de245e5a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:20.394807 6655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.369595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:35309 (local address 127.0.0.1:54936)
0927 22:57:20.384957 (+ 15362us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:20.384984 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:20.385152 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:20.387342 (+ 2190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:20.387348 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:20.387380 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:20.387614 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.387627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.389795 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.389801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.391735 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.391776 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.393828 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.393837 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.394095 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.394140 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 22:57:20.394266 (+ 126us) client_negotiation.cc:240] Negotiation successful
0927 22:57:20.394457 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15190,"thread_start_us":16823,"threads_started":1}
I20260927 22:57:20.394968 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.369950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54936 (local address 127.4.74.131:35309)
0927 22:57:20.386051 (+ 16101us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.386061 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.386102 (+ 41us) server_negotiation.cc:407] Connection header received
0927 22:57:20.386315 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.386329 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.386704 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.386963 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:20.387837 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.389581 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.392057 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.393601 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.393790 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.394040 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.394376 (+ 336us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.394563 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15838,"thread_start_us":15972,"threads_started":1}
I20260927 22:57:20.399514 6572 tablet_service.cc:1467] Processing CreateTablet for tablet d94ac5a8eb7541949ef388de245e5a39 (DEFAULT_TABLE table=test-table [id=15201661024347e9b2e6146b2b03ec99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:20.401384 6572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d94ac5a8eb7541949ef388de245e5a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:20.410954 6661 tablet_bootstrap.cc:492] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: Bootstrap starting.
I20260927 22:57:20.413723 6662 tablet_bootstrap.cc:492] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: Bootstrap starting.
I20260927 22:57:20.416527 6661 tablet_bootstrap.cc:654] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:20.418460 6662 tablet_bootstrap.cc:654] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:20.418968 6663 tablet_bootstrap.cc:492] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: Bootstrap starting.
I20260927 22:57:20.419287 6661 log.cc:824] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:20.420105 6662 log.cc:824] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:20.422645 6663 tablet_bootstrap.cc:654] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:20.424233 6663 log.cc:824] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:20.426640 6662 tablet_bootstrap.cc:492] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: No bootstrap required, opened a new log
I20260927 22:57:20.426759 6661 tablet_bootstrap.cc:492] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: No bootstrap required, opened a new log
I20260927 22:57:20.427016 6662 ts_tablet_manager.cc:1397] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260927 22:57:20.427227 6661 ts_tablet_manager.cc:1397] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.010s
I20260927 22:57:20.428790 6663 tablet_bootstrap.cc:492] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: No bootstrap required, opened a new log
I20260927 22:57:20.429141 6663 ts_tablet_manager.cc:1397] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260927 22:57:20.434473 6662 raft_consensus.cc:348] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.434921 6662 raft_consensus.cc:374] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:20.435070 6662 raft_consensus.cc:729] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513b026fe15b45ca944708db85567088, State: Initialized, Role: FOLLOWER
I20260927 22:57:20.435804 6662 consensus_queue.cc:260] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.437664 6662 ts_tablet_manager.cc:1428] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260927 22:57:20.440178 6661 raft_consensus.cc:348] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.440764 6661 raft_consensus.cc:374] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:20.440887 6661 raft_consensus.cc:729] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2313a7f01817402fbd824734bc8608fe, State: Initialized, Role: FOLLOWER
I20260927 22:57:20.441618 6663 raft_consensus.cc:348] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.441707 6661 consensus_queue.cc:260] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.442019 6663 raft_consensus.cc:374] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:20.442153 6663 raft_consensus.cc:729] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e819715daae426a9cc50cb06fef6b23, State: Initialized, Role: FOLLOWER
I20260927 22:57:20.442956 6663 consensus_queue.cc:260] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.443356 6502 heartbeater.cc:503] Master 127.4.74.190:39421 was elected leader, sending a full tablet report...
I20260927 22:57:20.444381 6661 ts_tablet_manager.cc:1428] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: Time spent starting tablet: real 0.017s user 0.014s sys 0.003s
I20260927 22:57:20.446267 6642 heartbeater.cc:503] Master 127.4.74.190:39421 was elected leader, sending a full tablet report...
I20260927 22:57:20.446592 6663 ts_tablet_manager.cc:1428] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: Time spent starting tablet: real 0.017s user 0.011s sys 0.006s
W20260927 22:57:20.516759 6643 tablet.cc:2369] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 22:57:20.604939 6363 tablet.cc:2369] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 22:57:20.670482 6503 tablet.cc:2369] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:57:20.681926 6667 raft_consensus.cc:482] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:20.682427 6667 raft_consensus.cc:504] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.685140 6667 leader_election.cc:288] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619), 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131:35309)
I20260927 22:57:20.694020 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.685576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:38055 (local address 127.4.74.130:38619)
0927 22:57:20.685810 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.685817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.686440 (+ 623us) server_negotiation.cc:407] Connection header received
0927 22:57:20.686716 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.686721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.686805 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.686920 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:20.687834 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.689140 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.691442 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.692778 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.692937 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.693009 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.693464 (+ 455us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.693620 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260927 22:57:20.694105 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.685412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:38619 (local address 127.4.74.129:38055)
0927 22:57:20.686285 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:20.686310 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:20.686497 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:20.687133 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:20.687139 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:20.687177 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:20.687619 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.687634 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.689320 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.689333 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.691240 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.691254 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.693037 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.693044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.693255 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.693302 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 22:57:20.693428 (+ 126us) client_negotiation.cc:240] Negotiation successful
0927 22:57:20.693595 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":28,"thread_start_us":438,"threads_started":1}
I20260927 22:57:20.694888 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.686735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:51847 (local address 127.4.74.131:35309)
0927 22:57:20.686946 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.686952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.687416 (+ 464us) server_negotiation.cc:407] Connection header received
0927 22:57:20.687590 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.687596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.687671 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.687766 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:20.688933 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.689964 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.692455 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.693588 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.693767 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.693842 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.694553 (+ 711us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.694657 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 22:57:20.695135 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.686405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:35309 (local address 127.4.74.129:51847)
0927 22:57:20.687289 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:20.687307 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:20.687432 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:20.688217 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:20.688224 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:20.688285 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:20.688730 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.688744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.690162 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.690167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.692242 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.692260 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.693873 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.693880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.694134 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.694174 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 22:57:20.694490 (+ 316us) client_negotiation.cc:240] Negotiation successful
0927 22:57:20.694667 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":1,"thread_start_us":442,"threads_started":1}
I20260927 22:57:20.695433 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d94ac5a8eb7541949ef388de245e5a39" candidate_uuid: "513b026fe15b45ca944708db85567088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313a7f01817402fbd824734bc8608fe" is_pre_election: true
I20260927 22:57:20.696086 6452 raft_consensus.cc:2454] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513b026fe15b45ca944708db85567088 in term 0.
I20260927 22:57:20.696470 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d94ac5a8eb7541949ef388de245e5a39" candidate_uuid: "513b026fe15b45ca944708db85567088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e819715daae426a9cc50cb06fef6b23" is_pre_election: true
I20260927 22:57:20.696877 6242 leader_election.cc:302] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313a7f01817402fbd824734bc8608fe, 513b026fe15b45ca944708db85567088; no voters:
I20260927 22:57:20.697225 6592 raft_consensus.cc:2454] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513b026fe15b45ca944708db85567088 in term 0.
I20260927 22:57:20.697773 6667 raft_consensus.cc:2781] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:20.697980 6667 raft_consensus.cc:482] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:20.698251 6667 raft_consensus.cc:3037] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:20.701396 6667 raft_consensus.cc:504] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.702178 6667 leader_election.cc:288] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [CANDIDATE]: Term 1 election: Requested vote from peers 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619), 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131:35309)
I20260927 22:57:20.703120 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d94ac5a8eb7541949ef388de245e5a39" candidate_uuid: "513b026fe15b45ca944708db85567088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313a7f01817402fbd824734bc8608fe"
I20260927 22:57:20.703358 6452 raft_consensus.cc:3037] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:20.703773 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d94ac5a8eb7541949ef388de245e5a39" candidate_uuid: "513b026fe15b45ca944708db85567088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e819715daae426a9cc50cb06fef6b23"
I20260927 22:57:20.704048 6592 raft_consensus.cc:3037] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:20.707196 6452 raft_consensus.cc:2454] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513b026fe15b45ca944708db85567088 in term 1.
I20260927 22:57:20.707505 6592 raft_consensus.cc:2454] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513b026fe15b45ca944708db85567088 in term 1.
I20260927 22:57:20.708130 6242 leader_election.cc:302] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313a7f01817402fbd824734bc8608fe, 513b026fe15b45ca944708db85567088; no voters:
I20260927 22:57:20.708550 6667 raft_consensus.cc:2781] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:20.709136 6667 raft_consensus.cc:686] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 1 LEADER]: Becoming Leader. State: Replica: 513b026fe15b45ca944708db85567088, State: Running, Role: LEADER
I20260927 22:57:20.710114 6667 consensus_queue.cc:237] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.716321 6155 catalog_manager.cc:5495] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 reported cstate change: term changed from 0 to 1, leader changed from <none> to 513b026fe15b45ca944708db85567088 (127.4.74.129). New cstate: current_term: 1 leader_uuid: "513b026fe15b45ca944708db85567088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } health_report { overall_health: UNKNOWN } } }
Sep 27 22:57:20 dist-test-slave-5pd3 krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549838, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:20.806802 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.790727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39421 (local address 127.0.0.1:56050)
0927 22:57:20.791374 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:20.791392 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:20.791652 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:20.792078 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:20.792082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:20.792886 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:20.793167 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.793176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.794298 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.794302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.795358 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.795369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.796249 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.796257 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.796392 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.797184 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:20.797205 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:20.800194 (+ 2989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:20.803612 (+ 3418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:20.803626 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:20.803634 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:20.804256 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:20.804746 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:20.804750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:20.804753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:20.804952 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:20.805568 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:20.805576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:20.805786 (+ 210us) client_negotiation.cc:769] Sending connection context
0927 22:57:20.805983 (+ 197us) client_negotiation.cc:240] Negotiation successful
0927 22:57:20.806328 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":350,"threads_started":1}
I20260927 22:57:20.806823 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.790674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.4.74.190:39421)
0927 22:57:20.790863 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.790868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.791516 (+ 648us) server_negotiation.cc:407] Connection header received
0927 22:57:20.791841 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.791845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.791922 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.792027 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:20.793359 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.794144 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.795523 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.796029 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.796139 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.800491 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:20.800498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:20.800501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:20.800543 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:20.803374 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:20.804429 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:20.804433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:20.804435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:20.804593 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:20.805087 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:20.805090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:20.805093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:20.805412 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:20.805538 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.806109 (+ 571us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.806477 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":59}
I20260927 22:57:20.813359 6155 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:56050:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 22:57:20.814406 6155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:20.822188 6292 tablet_service.cc:1467] Processing CreateTablet for tablet c04698d67d13463c8db736ff7cb3cfd1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d481346a82b4772b5040ade3ee460c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 22:57:20.822547 6572 tablet_service.cc:1467] Processing CreateTablet for tablet c04698d67d13463c8db736ff7cb3cfd1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d481346a82b4772b5040ade3ee460c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 22:57:20.823287 6572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04698d67d13463c8db736ff7cb3cfd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:20.823287 6292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04698d67d13463c8db736ff7cb3cfd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:20.822762 6432 tablet_service.cc:1467] Processing CreateTablet for tablet c04698d67d13463c8db736ff7cb3cfd1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d481346a82b4772b5040ade3ee460c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 22:57:20.823931 6432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04698d67d13463c8db736ff7cb3cfd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:20.830689 6661 tablet_bootstrap.cc:492] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: Bootstrap starting.
I20260927 22:57:20.830896 6662 tablet_bootstrap.cc:492] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: Bootstrap starting.
I20260927 22:57:20.831775 6663 tablet_bootstrap.cc:492] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: Bootstrap starting.
I20260927 22:57:20.832718 6661 tablet_bootstrap.cc:654] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:20.832914 6662 tablet_bootstrap.cc:654] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:20.833842 6663 tablet_bootstrap.cc:654] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:20.835940 6662 tablet_bootstrap.cc:492] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: No bootstrap required, opened a new log
I20260927 22:57:20.836117 6662 ts_tablet_manager.cc:1397] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260927 22:57:20.836648 6663 tablet_bootstrap.cc:492] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: No bootstrap required, opened a new log
I20260927 22:57:20.836830 6663 ts_tablet_manager.cc:1397] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260927 22:57:20.836985 6662 raft_consensus.cc:348] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.837247 6661 tablet_bootstrap.cc:492] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: No bootstrap required, opened a new log
I20260927 22:57:20.837270 6662 raft_consensus.cc:374] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:20.837354 6662 raft_consensus.cc:729] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513b026fe15b45ca944708db85567088, State: Initialized, Role: FOLLOWER
I20260927 22:57:20.837409 6661 ts_tablet_manager.cc:1397] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260927 22:57:20.837604 6662 consensus_queue.cc:260] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.837611 6663 raft_consensus.cc:348] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.837884 6663 raft_consensus.cc:374] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:20.837978 6663 raft_consensus.cc:729] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e819715daae426a9cc50cb06fef6b23, State: Initialized, Role: FOLLOWER
I20260927 22:57:20.838234 6663 consensus_queue.cc:260] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.838196 6667 tablet_replica.cc:439] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } } }
I20260927 22:57:20.838554 6667 tablet_replica.cc:442] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:20.838903 6663 ts_tablet_manager.cc:1428] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260927 22:57:20.838965 6669 tablet_replica.cc:439] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } } }
I20260927 22:57:20.839301 6669 tablet_replica.cc:442] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:20.839396 6661 raft_consensus.cc:348] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.839775 6661 raft_consensus.cc:374] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:20.839929 6662 ts_tablet_manager.cc:1428] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260927 22:57:20.839915 6661 raft_consensus.cc:729] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2313a7f01817402fbd824734bc8608fe, State: Initialized, Role: FOLLOWER
I20260927 22:57:20.840193 6661 consensus_queue.cc:260] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.841374 6661 ts_tablet_manager.cc:1428] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260927 22:57:20.841275 6668 tablet_replica.cc:439] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } } }
I20260927 22:57:20.842243 6668 tablet_replica.cc:442] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:20.977198 6669 raft_consensus.cc:482] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:20.977558 6669 raft_consensus.cc:504] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.979676 6669 leader_election.cc:288] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 513b026fe15b45ca944708db85567088 (127.4.74.129:34697), 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619)
I20260927 22:57:20.988889 6684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.980039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:38619 (local address 127.4.74.131:59615)
0927 22:57:20.981372 (+ 1333us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:20.981401 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:20.981703 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:20.982319 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:20.982329 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:20.982361 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:20.982780 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.982805 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.984295 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.984301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.986108 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.986125 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.987688 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.987698 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.988041 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.988086 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 22:57:20.988242 (+ 156us) client_negotiation.cc:240] Negotiation successful
0927 22:57:20.988446 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1136,"thread_start_us":1435,"threads_started":1}
I20260927 22:57:20.988943 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.980525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:59615 (local address 127.4.74.130:38619)
0927 22:57:20.980798 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.980809 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.981551 (+ 742us) server_negotiation.cc:407] Connection header received
0927 22:57:20.981922 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.981929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.982020 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.982090 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:20.983083 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.984085 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.986300 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.987331 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.987475 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.987609 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.988315 (+ 706us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.988499 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260927 22:57:20.989742 6683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.980163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:34697 (local address 127.4.74.131:49305)
0927 22:57:20.981881 (+ 1718us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:20.981911 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:20.982076 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:20.982496 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:20.982501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:20.982525 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:20.982990 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.983002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.984576 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.984582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.986568 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:20.986587 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.988663 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.988671 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:20.988895 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.988945 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 22:57:20.989077 (+ 132us) client_negotiation.cc:240] Negotiation successful
0927 22:57:20.989238 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1549,"mutex_wait_us":4,"thread_start_us":811,"threads_started":1}
I20260927 22:57:20.989873 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04698d67d13463c8db736ff7cb3cfd1" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313a7f01817402fbd824734bc8608fe" is_pre_election: true
I20260927 22:57:20.990100 6685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:20.980221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:49305 (local address 127.4.74.129:34697)
0927 22:57:20.982097 (+ 1876us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:20.982104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:20.982123 (+ 19us) server_negotiation.cc:407] Connection header received
0927 22:57:20.982202 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:20.982207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:20.982291 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:20.982417 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:20.983196 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.984352 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.986890 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:20.988333 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:20.988505 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:20.988586 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:20.989513 (+ 927us) server_negotiation.cc:299] Negotiation successful
0927 22:57:20.989704 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1737,"thread_start_us":1601,"threads_started":1}
I20260927 22:57:20.990227 6452 raft_consensus.cc:2454] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 0.
I20260927 22:57:20.990878 6522 leader_election.cc:302] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313a7f01817402fbd824734bc8608fe, 3e819715daae426a9cc50cb06fef6b23; no voters:
I20260927 22:57:20.991284 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04698d67d13463c8db736ff7cb3cfd1" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513b026fe15b45ca944708db85567088" is_pre_election: true
I20260927 22:57:20.991509 6669 raft_consensus.cc:2781] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:20.991715 6669 raft_consensus.cc:482] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:20.991832 6669 raft_consensus.cc:3037] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:20.991912 6316 raft_consensus.cc:2454] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 0.
I20260927 22:57:20.994323 6669 raft_consensus.cc:504] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:20.995039 6669 leader_election.cc:288] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 election: Requested vote from peers 513b026fe15b45ca944708db85567088 (127.4.74.129:34697), 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619)
I20260927 22:57:20.995750 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04698d67d13463c8db736ff7cb3cfd1" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313a7f01817402fbd824734bc8608fe"
I20260927 22:57:20.995869 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04698d67d13463c8db736ff7cb3cfd1" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513b026fe15b45ca944708db85567088"
I20260927 22:57:20.996107 6452 raft_consensus.cc:3037] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:20.996196 6316 raft_consensus.cc:3037] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:20.998719 6452 raft_consensus.cc:2454] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 1.
I20260927 22:57:20.998762 6316 raft_consensus.cc:2454] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 1.
I20260927 22:57:20.999329 6522 leader_election.cc:302] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313a7f01817402fbd824734bc8608fe, 3e819715daae426a9cc50cb06fef6b23; no voters:
I20260927 22:57:20.999728 6669 raft_consensus.cc:2781] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:21.000279 6669 raft_consensus.cc:686] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [term 1 LEADER]: Becoming Leader. State: Replica: 3e819715daae426a9cc50cb06fef6b23, State: Running, Role: LEADER
I20260927 22:57:21.001332 6669 consensus_queue.cc:237] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:21.005939 6686 tablet_replica.cc:439] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: TxnStatusTablet state changed. Reason: New leader 3e819715daae426a9cc50cb06fef6b23. Latest consensus state: current_term: 1 leader_uuid: "3e819715daae426a9cc50cb06fef6b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } } }
I20260927 22:57:21.006488 6686 tablet_replica.cc:442] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: This TxnStatusTablet replica's current role is: LEADER
I20260927 22:57:21.007323 6688 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 22:57:21.007674 6688 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 22:57:21.007965 6154 catalog_manager.cc:5495] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131). New cstate: current_term: 1 leader_uuid: "3e819715daae426a9cc50cb06fef6b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } health_report { overall_health: HEALTHY } } }
I20260927 22:57:21.043659 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:21.035172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.4.74.131:35309)
0927 22:57:21.035354 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:21.035358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:21.035927 (+ 569us) server_negotiation.cc:407] Connection header received
0927 22:57:21.036523 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:21.036552 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:21.036676 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:21.036879 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:21.037456 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.038411 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.040925 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.041431 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.041572 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:21.042261 (+ 689us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:21.042923 (+ 662us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:21.043054 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:21.043241 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 22:57:21.043318 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 22:57:21.043833 6679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:21.034998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:35309 (local address 127.0.0.1:54938)
0927 22:57:21.035746 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:21.035777 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:21.036131 (+ 354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:21.036940 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:21.036946 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:21.036964 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 22:57:21.037225 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:21.037237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.038589 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.038595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:21.040706 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:21.040727 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.041645 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.041653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:21.041861 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:21.041968 (+ 107us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 22:57:21.043073 (+ 1105us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 22:57:21.043097 (+ 24us) client_negotiation.cc:769] Sending connection context
0927 22:57:21.043207 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 22:57:21.043417 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264}
I20260927 22:57:21.052647 6316 raft_consensus.cc:1261] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Refusing update from remote peer 3e819715daae426a9cc50cb06fef6b23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:21.052659 6452 raft_consensus.cc:1261] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Refusing update from remote peer 3e819715daae426a9cc50cb06fef6b23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:21.053686 6669 consensus_queue.cc:1035] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:21.054073 6686 consensus_queue.cc:1035] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:21.065949 6667 tablet_replica.cc:439] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: TxnStatusTablet state changed. Reason: New leader 3e819715daae426a9cc50cb06fef6b23. Latest consensus state: current_term: 1 leader_uuid: "3e819715daae426a9cc50cb06fef6b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } } }
I20260927 22:57:21.066411 6667 tablet_replica.cc:442] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:21.070317 6692 mvcc.cc:205] Tried to move back new op lower bound from 7334092148940763136 to 7334092148747083776. Current Snapshot: MvccSnapshot[applied={T|T < 7334092148940763136}]
I20260927 22:57:21.072857 6691 mvcc.cc:205] Tried to move back new op lower bound from 7334092148940763136 to 7334092148747083776. Current Snapshot: MvccSnapshot[applied={T|T < 7334092148940763136}]
I20260927 22:57:21.075090 6669 tablet_replica.cc:439] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e819715daae426a9cc50cb06fef6b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } } }
I20260927 22:57:21.075472 6669 tablet_replica.cc:442] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: This TxnStatusTablet replica's current role is: LEADER
I20260927 22:57:21.084900 6667 tablet_replica.cc:439] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e819715daae426a9cc50cb06fef6b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } } }
I20260927 22:57:21.085280 6667 tablet_replica.cc:442] T c04698d67d13463c8db736ff7cb3cfd1 P 513b026fe15b45ca944708db85567088: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:21.086509 6668 tablet_replica.cc:439] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: TxnStatusTablet state changed. Reason: New leader 3e819715daae426a9cc50cb06fef6b23. Latest consensus state: current_term: 1 leader_uuid: "3e819715daae426a9cc50cb06fef6b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } } }
I20260927 22:57:21.086882 6668 tablet_replica.cc:442] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:21.088587 6669 tablet_replica.cc:439] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e819715daae426a9cc50cb06fef6b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } } }
I20260927 22:57:21.088961 6669 tablet_replica.cc:442] T c04698d67d13463c8db736ff7cb3cfd1 P 3e819715daae426a9cc50cb06fef6b23: This TxnStatusTablet replica's current role is: LEADER
I20260927 22:57:21.092521 6693 mvcc.cc:205] Tried to move back new op lower bound from 7334092148940763136 to 7334092148747083776. Current Snapshot: MvccSnapshot[applied={T|T < 7334092148940763136}]
I20260927 22:57:21.097848 6668 tablet_replica.cc:439] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e819715daae426a9cc50cb06fef6b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } } }
I20260927 22:57:21.098176 6668 tablet_replica.cc:442] T c04698d67d13463c8db736ff7cb3cfd1 P 2313a7f01817402fbd824734bc8608fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:57:21.116550 6685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:21.093179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36358 (local address 127.4.74.129:34697)
0927 22:57:21.093395 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:21.093401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:21.108380 (+ 14979us) server_negotiation.cc:407] Connection header received
0927 22:57:21.108785 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:21.108791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:21.108894 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:21.109051 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:21.109732 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.110932 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.113156 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.113974 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.114185 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:21.114731 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:21.115723 (+ 992us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:21.115902 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:21.116044 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 22:57:21.116166 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 22:57:21.135632 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:21.130111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:53539 (local address 127.4.74.131:35309)
0927 22:57:21.130352 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:21.130359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:21.130614 (+ 255us) server_negotiation.cc:407] Connection header received
0927 22:57:21.130842 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:21.130854 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:21.130942 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:21.131042 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:21.131536 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.132325 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.133714 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.134181 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.134330 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:21.134779 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:21.134893 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:21.135015 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:21.135160 (+ 145us) server_negotiation.cc:299] Negotiation successful
0927 22:57:21.135234 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 22:57:21.135700 6704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:21.129929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:35309 (local address 127.4.74.129:53539)
0927 22:57:21.130504 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:21.130524 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:21.130695 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:21.131068 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:21.131073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:21.131085 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 22:57:21.131395 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:21.131404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.132467 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.132475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:21.133568 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:21.133580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.134368 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.134376 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:21.134507 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:21.134566 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 22:57:21.135030 (+ 464us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 22:57:21.135044 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 22:57:21.135136 (+ 92us) client_negotiation.cc:240] Negotiation successful
0927 22:57:21.135281 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":432,"threads_started":1}
I20260927 22:57:21.148013 6592 raft_consensus.cc:1261] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Refusing update from remote peer 513b026fe15b45ca944708db85567088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:21.148696 6451 raft_consensus.cc:1261] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Refusing update from remote peer 513b026fe15b45ca944708db85567088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:21.149161 6672 consensus_queue.cc:1035] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:21.150029 6667 consensus_queue.cc:1035] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:21.198132 6685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:21.189617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:34355 (local address 127.4.74.129:34697)
0927 22:57:21.189860 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:21.189868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:21.190756 (+ 888us) server_negotiation.cc:407] Connection header received
0927 22:57:21.191040 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:21.191055 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:21.191183 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:21.191305 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:21.192099 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.193379 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.195463 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.196264 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.196425 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:21.196955 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:21.197085 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:21.197190 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:21.197524 (+ 334us) server_negotiation.cc:299] Negotiation successful
0927 22:57:21.197828 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 22:57:21.198251 6708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:21.189668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:34697 (local address 127.4.74.131:34355)
0927 22:57:21.190659 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:21.190680 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:21.190850 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:21.191333 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:21.191338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:21.191351 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 22:57:21.191913 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:21.191924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.193555 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.193560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:21.195289 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:21.195302 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.196482 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.196491 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:21.196698 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:21.196767 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 22:57:21.197190 (+ 423us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 22:57:21.197203 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 22:57:21.197503 (+ 300us) client_negotiation.cc:240] Negotiation successful
0927 22:57:21.197776 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":530,"threads_started":1}
I20260927 22:57:21.253458 6155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:21.253837 6155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:21.258919 6155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c: Sending DeleteTablet for 3 replicas of tablet d94ac5a8eb7541949ef388de245e5a39
I20260927 22:57:21.260577 6292 tablet_service.cc:1514] Processing DeleteTablet for tablet d94ac5a8eb7541949ef388de245e5a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:21 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:36346
I20260927 22:57:21.260833 6432 tablet_service.cc:1514] Processing DeleteTablet for tablet d94ac5a8eb7541949ef388de245e5a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:21 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:45788
I20260927 22:57:21.264761 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet d94ac5a8eb7541949ef388de245e5a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:21 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:54936
I20260927 22:57:21.266207 6155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:57:21.267360 6155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:21.273623 6712 tablet_replica.cc:330] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: stopping tablet replica
I20260927 22:57:21.274499 6712 raft_consensus.cc:2229] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:57:21.275293 6712 raft_consensus.cc:2258] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:21.277550 6712 ts_tablet_manager.cc:1905] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:21.281602 6292 tablet_service.cc:1467] Processing CreateTablet for tablet 3af82d3a172d457e890ff1f6d502b5ed (DEFAULT_TABLE table=test-table [id=16d65223320641b6aa828b31471c540c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:21.282241 6292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3af82d3a172d457e890ff1f6d502b5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:21.287811 6712 ts_tablet_manager.cc:1918] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:57:21.288311 6712 log.cc:1196] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/d94ac5a8eb7541949ef388de245e5a39
I20260927 22:57:21.289216 6431 tablet_service.cc:1467] Processing CreateTablet for tablet 3af82d3a172d457e890ff1f6d502b5ed (DEFAULT_TABLE table=test-table [id=16d65223320641b6aa828b31471c540c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:21.289507 6571 tablet_service.cc:1467] Processing CreateTablet for tablet 3af82d3a172d457e890ff1f6d502b5ed (DEFAULT_TABLE table=test-table [id=16d65223320641b6aa828b31471c540c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:21.289889 6431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3af82d3a172d457e890ff1f6d502b5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:21.290215 6571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3af82d3a172d457e890ff1f6d502b5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:21.306036 6663 tablet_bootstrap.cc:492] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23: Bootstrap starting.
I20260927 22:57:21.307164 6662 tablet_bootstrap.cc:492] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088: Bootstrap starting.
I20260927 22:57:21.308619 6712 ts_tablet_manager.cc:1939] T d94ac5a8eb7541949ef388de245e5a39 P 513b026fe15b45ca944708db85567088: Deleting consensus metadata
I20260927 22:57:21.309713 6662 tablet_bootstrap.cc:654] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:21.311372 6142 catalog_manager.cc:4841] TS 513b026fe15b45ca944708db85567088 (127.4.74.129:34697): tablet d94ac5a8eb7541949ef388de245e5a39 (table test-table [id=15201661024347e9b2e6146b2b03ec99]) successfully deleted
I20260927 22:57:21.312247 6661 tablet_bootstrap.cc:492] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe: Bootstrap starting.
I20260927 22:57:21.314705 6661 tablet_bootstrap.cc:654] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:21.319027 6713 tablet_replica.cc:330] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: stopping tablet replica
I20260927 22:57:21.319068 6663 tablet_bootstrap.cc:654] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:21.319697 6713 raft_consensus.cc:2229] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:21.320112 6713 raft_consensus.cc:2258] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:21.320500 6662 tablet_bootstrap.cc:492] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088: No bootstrap required, opened a new log
I20260927 22:57:21.320856 6662 ts_tablet_manager.cc:1397] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260927 22:57:21.322288 6662 raft_consensus.cc:348] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:21.322854 6662 raft_consensus.cc:374] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:21.323016 6662 raft_consensus.cc:729] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513b026fe15b45ca944708db85567088, State: Initialized, Role: FOLLOWER
I20260927 22:57:21.323577 6662 consensus_queue.cc:260] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:21.329702 6714 tablet_replica.cc:330] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: stopping tablet replica
I20260927 22:57:21.329962 6662 ts_tablet_manager.cc:1428] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260927 22:57:21.330865 6714 raft_consensus.cc:2229] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:21.331459 6714 raft_consensus.cc:2258] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:21.329173 6661 tablet_bootstrap.cc:492] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe: No bootstrap required, opened a new log
I20260927 22:57:21.336321 6661 ts_tablet_manager.cc:1397] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.005s
I20260927 22:57:21.337046 6663 tablet_bootstrap.cc:492] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23: No bootstrap required, opened a new log
I20260927 22:57:21.337309 6663 ts_tablet_manager.cc:1397] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.006s
I20260927 22:57:21.337560 6661 raft_consensus.cc:348] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:21.337888 6661 raft_consensus.cc:374] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:21.338042 6661 raft_consensus.cc:729] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2313a7f01817402fbd824734bc8608fe, State: Initialized, Role: FOLLOWER
I20260927 22:57:21.338348 6663 raft_consensus.cc:348] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:21.338389 6661 consensus_queue.cc:260] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:21.338725 6663 raft_consensus.cc:374] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:21.338824 6663 raft_consensus.cc:729] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e819715daae426a9cc50cb06fef6b23, State: Initialized, Role: FOLLOWER
I20260927 22:57:21.339118 6663 consensus_queue.cc:260] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:21.340086 6661 ts_tablet_manager.cc:1428] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260927 22:57:21.342730 6713 ts_tablet_manager.cc:1905] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:21.346755 6663 ts_tablet_manager.cc:1428] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260927 22:57:21.350013 6714 ts_tablet_manager.cc:1905] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:21.353684 6713 ts_tablet_manager.cc:1918] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:57:21.353987 6713 log.cc:1196] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/d94ac5a8eb7541949ef388de245e5a39
I20260927 22:57:21.354602 6713 ts_tablet_manager.cc:1939] T d94ac5a8eb7541949ef388de245e5a39 P 2313a7f01817402fbd824734bc8608fe: Deleting consensus metadata
I20260927 22:57:21.356849 6139 catalog_manager.cc:4841] TS 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619): tablet d94ac5a8eb7541949ef388de245e5a39 (table test-table [id=15201661024347e9b2e6146b2b03ec99]) successfully deleted
I20260927 22:57:21.359733 6714 ts_tablet_manager.cc:1918] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:57:21.359937 6714 log.cc:1196] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/d94ac5a8eb7541949ef388de245e5a39
I20260927 22:57:21.360513 6714 ts_tablet_manager.cc:1939] T d94ac5a8eb7541949ef388de245e5a39 P 3e819715daae426a9cc50cb06fef6b23: Deleting consensus metadata
I20260927 22:57:21.363858 6139 catalog_manager.cc:4841] TS 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131:35309): tablet d94ac5a8eb7541949ef388de245e5a39 (table test-table [id=15201661024347e9b2e6146b2b03ec99]) successfully deleted
I20260927 22:57:21.647395 6686 raft_consensus.cc:482] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:21.647765 6686 raft_consensus.cc:504] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:21.648684 6686 leader_election.cc:288] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 513b026fe15b45ca944708db85567088 (127.4.74.129:34697), 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619)
I20260927 22:57:21.649384 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3af82d3a172d457e890ff1f6d502b5ed" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313a7f01817402fbd824734bc8608fe" is_pre_election: true
I20260927 22:57:21.649497 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3af82d3a172d457e890ff1f6d502b5ed" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513b026fe15b45ca944708db85567088" is_pre_election: true
I20260927 22:57:21.649703 6452 raft_consensus.cc:2454] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 0.
I20260927 22:57:21.649746 6316 raft_consensus.cc:2454] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 0.
I20260927 22:57:21.650257 6522 leader_election.cc:302] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2313a7f01817402fbd824734bc8608fe, 3e819715daae426a9cc50cb06fef6b23, 513b026fe15b45ca944708db85567088; no voters:
I20260927 22:57:21.650611 6686 raft_consensus.cc:2781] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:21.650717 6686 raft_consensus.cc:482] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:21.650847 6686 raft_consensus.cc:3037] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:21.652875 6686 raft_consensus.cc:504] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:21.653513 6686 leader_election.cc:288] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 election: Requested vote from peers 513b026fe15b45ca944708db85567088 (127.4.74.129:34697), 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619)
I20260927 22:57:21.654098 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3af82d3a172d457e890ff1f6d502b5ed" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513b026fe15b45ca944708db85567088"
I20260927 22:57:21.654313 6316 raft_consensus.cc:3037] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:21.654186 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3af82d3a172d457e890ff1f6d502b5ed" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313a7f01817402fbd824734bc8608fe"
I20260927 22:57:21.654533 6452 raft_consensus.cc:3037] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:21.656885 6316 raft_consensus.cc:2454] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 1.
I20260927 22:57:21.657426 6525 leader_election.cc:302] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e819715daae426a9cc50cb06fef6b23, 513b026fe15b45ca944708db85567088; no voters:
I20260927 22:57:21.657698 6686 raft_consensus.cc:2781] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:21.657897 6686 raft_consensus.cc:686] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 1 LEADER]: Becoming Leader. State: Replica: 3e819715daae426a9cc50cb06fef6b23, State: Running, Role: LEADER
I20260927 22:57:21.658155 6686 consensus_queue.cc:237] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:21.658248 6452 raft_consensus.cc:2454] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 1.
I20260927 22:57:21.661645 6154 catalog_manager.cc:5495] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131). New cstate: current_term: 1 leader_uuid: "3e819715daae426a9cc50cb06fef6b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } health_report { overall_health: HEALTHY } } }
I20260927 22:57:21.709661 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:21.702626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54940 (local address 127.4.74.131:35309)
0927 22:57:21.703499 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:21.703507 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:21.703535 (+ 28us) server_negotiation.cc:407] Connection header received
0927 22:57:21.703683 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:21.703689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:21.703810 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:21.703950 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:21.704424 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.705757 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.707302 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.708068 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.708226 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:21.708715 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:21.708885 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:21.708983 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:21.709184 (+ 201us) server_negotiation.cc:299] Negotiation successful
0927 22:57:21.709285 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":590,"threads_started":1}
I20260927 22:57:21.725996 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:21.720769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:54985 (local address 127.4.74.131:35309)
0927 22:57:21.720988 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:21.720996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:21.721394 (+ 398us) server_negotiation.cc:407] Connection header received
0927 22:57:21.721589 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:21.721594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:21.721659 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:21.721754 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:21.722194 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.722935 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.724315 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.724713 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.724850 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:21.725227 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:21.725326 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:21.725416 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:21.725557 (+ 141us) server_negotiation.cc:299] Negotiation successful
0927 22:57:21.725627 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":48}
I20260927 22:57:21.726002 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:21.720724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:35309 (local address 127.4.74.131:54985)
0927 22:57:21.721287 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:21.721305 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:21.721469 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:21.721794 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:21.721798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:21.721811 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 22:57:21.722072 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:21.722081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.723063 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.723072 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:21.724171 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:21.724191 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:21.724864 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:21.724870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:21.725039 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:21.725074 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 22:57:21.725425 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 22:57:21.725437 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 22:57:21.725529 (+ 92us) client_negotiation.cc:240] Negotiation successful
0927 22:57:21.725663 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":65,"thread_start_us":286,"threads_started":1}
I20260927 22:57:21.737463 6452 raft_consensus.cc:1261] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Refusing update from remote peer 3e819715daae426a9cc50cb06fef6b23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:21.737498 6316 raft_consensus.cc:1261] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Refusing update from remote peer 3e819715daae426a9cc50cb06fef6b23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:21.738229 6698 consensus_queue.cc:1035] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:21.738705 6698 consensus_queue.cc:1035] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 22:57:21.763767 6691 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 22:57:23.392416 6153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:23.392715 6153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:23.398106 6153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c: Sending DeleteTablet for 3 replicas of tablet 3af82d3a172d457e890ff1f6d502b5ed
I20260927 22:57:23.399185 6292 tablet_service.cc:1514] Processing DeleteTablet for tablet 3af82d3a172d457e890ff1f6d502b5ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:23 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:36346
I20260927 22:57:23.399236 6432 tablet_service.cc:1514] Processing DeleteTablet for tablet 3af82d3a172d457e890ff1f6d502b5ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:23 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:45788
I20260927 22:57:23.399266 6571 tablet_service.cc:1514] Processing DeleteTablet for tablet 3af82d3a172d457e890ff1f6d502b5ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:23 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:54936
I20260927 22:57:23.412246 6751 tablet_replica.cc:330] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088: stopping tablet replica
I20260927 22:57:23.412580 6751 raft_consensus.cc:2229] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:23.412613 6750 tablet_replica.cc:330] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe: stopping tablet replica
I20260927 22:57:23.412700 6752 tablet_replica.cc:330] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23: stopping tablet replica
I20260927 22:57:23.412904 6750 raft_consensus.cc:2229] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:23.412932 6752 raft_consensus.cc:2229] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:57:23.413015 6751 raft_consensus.cc:2258] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:23.413218 6750 raft_consensus.cc:2258] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:23.413743 6752 raft_consensus.cc:2258] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:23.414326 6750 ts_tablet_manager.cc:1905] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:23.414412 6751 ts_tablet_manager.cc:1905] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:23.415234 6752 ts_tablet_manager.cc:1905] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:23.421460 6750 ts_tablet_manager.cc:1918] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 22:57:23.421660 6750 log.cc:1196] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/3af82d3a172d457e890ff1f6d502b5ed
I20260927 22:57:23.422154 6750 ts_tablet_manager.cc:1939] T 3af82d3a172d457e890ff1f6d502b5ed P 2313a7f01817402fbd824734bc8608fe: Deleting consensus metadata
I20260927 22:57:23.422374 6752 ts_tablet_manager.cc:1918] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 22:57:23.422564 6752 log.cc:1196] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/3af82d3a172d457e890ff1f6d502b5ed
I20260927 22:57:23.423053 6752 ts_tablet_manager.cc:1939] T 3af82d3a172d457e890ff1f6d502b5ed P 3e819715daae426a9cc50cb06fef6b23: Deleting consensus metadata
I20260927 22:57:23.423712 6139 catalog_manager.cc:4841] TS 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619): tablet 3af82d3a172d457e890ff1f6d502b5ed (table test-table [id=16d65223320641b6aa828b31471c540c]) successfully deleted
I20260927 22:57:23.423727 6751 ts_tablet_manager.cc:1918] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 22:57:23.423918 6751 log.cc:1196] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/3af82d3a172d457e890ff1f6d502b5ed
I20260927 22:57:23.424609 6751 ts_tablet_manager.cc:1939] T 3af82d3a172d457e890ff1f6d502b5ed P 513b026fe15b45ca944708db85567088: Deleting consensus metadata
I20260927 22:57:23.424526 6139 catalog_manager.cc:4841] TS 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131:35309): tablet 3af82d3a172d457e890ff1f6d502b5ed (table test-table [id=16d65223320641b6aa828b31471c540c]) successfully deleted
I20260927 22:57:23.426029 6142 catalog_manager.cc:4841] TS 513b026fe15b45ca944708db85567088 (127.4.74.129:34697): tablet 3af82d3a172d457e890ff1f6d502b5ed (table test-table [id=16d65223320641b6aa828b31471c540c]) successfully deleted
Sep 27 22:57:23 dist-test-slave-5pd3 krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549843, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 22:57:23 dist-test-slave-5pd3 krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549843, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:23.465612 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:23.449723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.4.74.190:39421)
0927 22:57:23.450453 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:23.450459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:23.450489 (+ 30us) server_negotiation.cc:407] Connection header received
0927 22:57:23.450611 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:23.450616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:23.450704 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:23.450828 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:23.452359 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:23.453233 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:23.454515 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:23.454943 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:23.455043 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:23.459246 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:23.459252 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:23.459256 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:23.459298 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:23.462335 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:23.463237 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:23.463241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:23.463243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:23.463384 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:23.463917 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:23.463921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:23.463923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:23.464256 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:23.464446 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:23.464970 (+ 524us) server_negotiation.cc:299] Negotiation successful
0927 22:57:23.465174 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":355,"threads_started":1}
I20260927 22:57:23.470091 6153 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:57:23.470875 6153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:23.479858 6292 tablet_service.cc:1467] Processing CreateTablet for tablet 5002701d242144d9a3fccbbf85f5eb4e (DEFAULT_TABLE table=test-table [id=7a59c91b0d654eeb92ebfd2462ac9cfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:23.480039 6432 tablet_service.cc:1467] Processing CreateTablet for tablet 5002701d242144d9a3fccbbf85f5eb4e (DEFAULT_TABLE table=test-table [id=7a59c91b0d654eeb92ebfd2462ac9cfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:23.480073 6571 tablet_service.cc:1467] Processing CreateTablet for tablet 5002701d242144d9a3fccbbf85f5eb4e (DEFAULT_TABLE table=test-table [id=7a59c91b0d654eeb92ebfd2462ac9cfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:23.480479 6292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5002701d242144d9a3fccbbf85f5eb4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:23.480518 6432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5002701d242144d9a3fccbbf85f5eb4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:23.480618 6571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5002701d242144d9a3fccbbf85f5eb4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:23.487428 6761 tablet_bootstrap.cc:492] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23: Bootstrap starting.
I20260927 22:57:23.488955 6762 tablet_bootstrap.cc:492] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe: Bootstrap starting.
I20260927 22:57:23.490473 6761 tablet_bootstrap.cc:654] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:23.491039 6762 tablet_bootstrap.cc:654] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:23.492722 6763 tablet_bootstrap.cc:492] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088: Bootstrap starting.
I20260927 22:57:23.493948 6761 tablet_bootstrap.cc:492] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23: No bootstrap required, opened a new log
I20260927 22:57:23.494200 6761 ts_tablet_manager.cc:1397] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260927 22:57:23.494339 6762 tablet_bootstrap.cc:492] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe: No bootstrap required, opened a new log
I20260927 22:57:23.494541 6762 ts_tablet_manager.cc:1397] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260927 22:57:23.494805 6763 tablet_bootstrap.cc:654] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:23.495525 6762 raft_consensus.cc:348] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.495857 6762 raft_consensus.cc:374] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:23.495657 6761 raft_consensus.cc:348] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.495960 6761 raft_consensus.cc:374] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:23.495980 6762 raft_consensus.cc:729] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2313a7f01817402fbd824734bc8608fe, State: Initialized, Role: FOLLOWER
I20260927 22:57:23.496106 6761 raft_consensus.cc:729] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e819715daae426a9cc50cb06fef6b23, State: Initialized, Role: FOLLOWER
I20260927 22:57:23.496316 6762 consensus_queue.cc:260] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.496788 6761 consensus_queue.cc:260] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.497963 6761 ts_tablet_manager.cc:1428] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260927 22:57:23.498624 6762 ts_tablet_manager.cc:1428] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260927 22:57:23.500581 6763 tablet_bootstrap.cc:492] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088: No bootstrap required, opened a new log
I20260927 22:57:23.500787 6763 ts_tablet_manager.cc:1397] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260927 22:57:23.501802 6763 raft_consensus.cc:348] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.502099 6763 raft_consensus.cc:374] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:23.502266 6763 raft_consensus.cc:729] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513b026fe15b45ca944708db85567088, State: Initialized, Role: FOLLOWER
I20260927 22:57:23.502600 6763 consensus_queue.cc:260] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.503538 6763 ts_tablet_manager.cc:1428] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 22:57:23.569770 6737 raft_consensus.cc:482] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:23.570072 6737 raft_consensus.cc:504] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.570840 6737 leader_election.cc:288] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 513b026fe15b45ca944708db85567088 (127.4.74.129:34697), 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619)
I20260927 22:57:23.571591 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5002701d242144d9a3fccbbf85f5eb4e" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513b026fe15b45ca944708db85567088" is_pre_election: true
I20260927 22:57:23.571722 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5002701d242144d9a3fccbbf85f5eb4e" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313a7f01817402fbd824734bc8608fe" is_pre_election: true
I20260927 22:57:23.571892 6316 raft_consensus.cc:2454] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 0.
I20260927 22:57:23.572203 6452 raft_consensus.cc:2454] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 0.
I20260927 22:57:23.572603 6525 leader_election.cc:302] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e819715daae426a9cc50cb06fef6b23, 513b026fe15b45ca944708db85567088; no voters:
I20260927 22:57:23.573040 6737 raft_consensus.cc:2781] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:23.573190 6737 raft_consensus.cc:482] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:23.573295 6737 raft_consensus.cc:3037] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:23.575790 6737 raft_consensus.cc:504] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.576642 6737 leader_election.cc:288] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 election: Requested vote from peers 513b026fe15b45ca944708db85567088 (127.4.74.129:34697), 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619)
I20260927 22:57:23.577270 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5002701d242144d9a3fccbbf85f5eb4e" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513b026fe15b45ca944708db85567088"
I20260927 22:57:23.577280 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5002701d242144d9a3fccbbf85f5eb4e" candidate_uuid: "3e819715daae426a9cc50cb06fef6b23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313a7f01817402fbd824734bc8608fe"
I20260927 22:57:23.577521 6316 raft_consensus.cc:3037] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:23.577554 6452 raft_consensus.cc:3037] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:23.579811 6316 raft_consensus.cc:2454] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 1.
I20260927 22:57:23.579829 6452 raft_consensus.cc:2454] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e819715daae426a9cc50cb06fef6b23 in term 1.
I20260927 22:57:23.580345 6525 leader_election.cc:302] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e819715daae426a9cc50cb06fef6b23, 513b026fe15b45ca944708db85567088; no voters:
I20260927 22:57:23.580716 6737 raft_consensus.cc:2781] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:23.580909 6737 raft_consensus.cc:686] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 1 LEADER]: Becoming Leader. State: Replica: 3e819715daae426a9cc50cb06fef6b23, State: Running, Role: LEADER
I20260927 22:57:23.581218 6737 consensus_queue.cc:237] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.584570 6154 catalog_manager.cc:5495] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131). New cstate: current_term: 1 leader_uuid: "3e819715daae426a9cc50cb06fef6b23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } health_report { overall_health: HEALTHY } } }
I20260927 22:57:23.642655 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:23.636331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54944 (local address 127.4.74.131:35309)
0927 22:57:23.637205 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:23.637212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:23.637243 (+ 31us) server_negotiation.cc:407] Connection header received
0927 22:57:23.637327 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:23.637332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:23.637424 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:23.637557 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:23.638069 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:23.638844 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:23.640662 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:23.641096 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:23.641196 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:23.641760 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:23.641907 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:23.641987 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:23.642201 (+ 214us) server_negotiation.cc:299] Negotiation successful
0927 22:57:23.642265 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":457,"threads_started":1}
I20260927 22:57:23.653951 6316 raft_consensus.cc:1261] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Refusing update from remote peer 3e819715daae426a9cc50cb06fef6b23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:23.654206 6452 raft_consensus.cc:1261] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Refusing update from remote peer 3e819715daae426a9cc50cb06fef6b23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:23.654663 6745 consensus_queue.cc:1035] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:23.655158 6770 consensus_queue.cc:1035] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:23.707876 6153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:23.708252 6153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:23.711864 6153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c: Sending DeleteTablet for 3 replicas of tablet 5002701d242144d9a3fccbbf85f5eb4e
I20260927 22:57:23.713020 6571 tablet_service.cc:1514] Processing DeleteTablet for tablet 5002701d242144d9a3fccbbf85f5eb4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:23 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:54936
I20260927 22:57:23.713034 6432 tablet_service.cc:1514] Processing DeleteTablet for tablet 5002701d242144d9a3fccbbf85f5eb4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:23 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:45788
I20260927 22:57:23.712994 6292 tablet_service.cc:1514] Processing DeleteTablet for tablet 5002701d242144d9a3fccbbf85f5eb4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:23 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:36346
I20260927 22:57:23.713338 6750 tablet_replica.cc:330] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe: stopping tablet replica
I20260927 22:57:23.713374 6751 tablet_replica.cc:330] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088: stopping tablet replica
I20260927 22:57:23.713464 6752 tablet_replica.cc:330] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23: stopping tablet replica
I20260927 22:57:23.713564 6750 raft_consensus.cc:2229] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:23.713629 6751 raft_consensus.cc:2229] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:23.713690 6752 raft_consensus.cc:2229] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:57:23.713792 6750 raft_consensus.cc:2258] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:23.713886 6751 raft_consensus.cc:2258] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:23.714051 6752 raft_consensus.cc:2258] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:23.715170 6750 ts_tablet_manager.cc:1905] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:23.715359 6751 ts_tablet_manager.cc:1905] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:23.715277 6153 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 22:57:23.715914 6752 ts_tablet_manager.cc:1905] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 22:57:23.716183 6153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:23.722479 6751 ts_tablet_manager.cc:1918] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:57:23.722687 6751 log.cc:1196] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/5002701d242144d9a3fccbbf85f5eb4e
I20260927 22:57:23.722652 6750 ts_tablet_manager.cc:1918] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:57:23.722918 6750 log.cc:1196] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/5002701d242144d9a3fccbbf85f5eb4e
I20260927 22:57:23.723167 6751 ts_tablet_manager.cc:1939] T 5002701d242144d9a3fccbbf85f5eb4e P 513b026fe15b45ca944708db85567088: Deleting consensus metadata
I20260927 22:57:23.723483 6750 ts_tablet_manager.cc:1939] T 5002701d242144d9a3fccbbf85f5eb4e P 2313a7f01817402fbd824734bc8608fe: Deleting consensus metadata
I20260927 22:57:23.723928 6292 tablet_service.cc:1467] Processing CreateTablet for tablet d2017c1f62cf4eac996414955aa09b3a (DEFAULT_TABLE table=test-table [id=fa583a4dcbbe4a8c8b138054ad444c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:23.724105 6571 tablet_service.cc:1467] Processing CreateTablet for tablet d2017c1f62cf4eac996414955aa09b3a (DEFAULT_TABLE table=test-table [id=fa583a4dcbbe4a8c8b138054ad444c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:23.724445 6292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2017c1f62cf4eac996414955aa09b3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:23.724485 6571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2017c1f62cf4eac996414955aa09b3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:23.724896 6139 catalog_manager.cc:4841] TS 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619): tablet 5002701d242144d9a3fccbbf85f5eb4e (table test-table [id=7a59c91b0d654eeb92ebfd2462ac9cfd]) successfully deleted
I20260927 22:57:23.727977 6432 tablet_service.cc:1467] Processing CreateTablet for tablet d2017c1f62cf4eac996414955aa09b3a (DEFAULT_TABLE table=test-table [id=fa583a4dcbbe4a8c8b138054ad444c90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:23.728085 6752 ts_tablet_manager.cc:1918] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:57:23.728295 6752 log.cc:1196] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/5002701d242144d9a3fccbbf85f5eb4e
I20260927 22:57:23.728457 6432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2017c1f62cf4eac996414955aa09b3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:23.728791 6752 ts_tablet_manager.cc:1939] T 5002701d242144d9a3fccbbf85f5eb4e P 3e819715daae426a9cc50cb06fef6b23: Deleting consensus metadata
I20260927 22:57:23.729984 6139 catalog_manager.cc:4841] TS 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131:35309): tablet 5002701d242144d9a3fccbbf85f5eb4e (table test-table [id=7a59c91b0d654eeb92ebfd2462ac9cfd]) successfully deleted
I20260927 22:57:23.730479 6142 catalog_manager.cc:4841] TS 513b026fe15b45ca944708db85567088 (127.4.74.129:34697): tablet 5002701d242144d9a3fccbbf85f5eb4e (table test-table [id=7a59c91b0d654eeb92ebfd2462ac9cfd]) successfully deleted
I20260927 22:57:23.730820 6761 tablet_bootstrap.cc:492] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23: Bootstrap starting.
I20260927 22:57:23.732470 6761 tablet_bootstrap.cc:654] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:23.733140 6763 tablet_bootstrap.cc:492] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088: Bootstrap starting.
I20260927 22:57:23.734951 6761 tablet_bootstrap.cc:492] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23: No bootstrap required, opened a new log
I20260927 22:57:23.735013 6763 tablet_bootstrap.cc:654] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:23.735110 6761 ts_tablet_manager.cc:1397] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260927 22:57:23.735895 6762 tablet_bootstrap.cc:492] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe: Bootstrap starting.
I20260927 22:57:23.735903 6761 raft_consensus.cc:348] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.736150 6761 raft_consensus.cc:374] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:23.736249 6761 raft_consensus.cc:729] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e819715daae426a9cc50cb06fef6b23, State: Initialized, Role: FOLLOWER
I20260927 22:57:23.736532 6761 consensus_queue.cc:260] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.737331 6761 ts_tablet_manager.cc:1428] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260927 22:57:23.737821 6762 tablet_bootstrap.cc:654] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:23.738778 6763 tablet_bootstrap.cc:492] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088: No bootstrap required, opened a new log
I20260927 22:57:23.739027 6763 ts_tablet_manager.cc:1397] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260927 22:57:23.739902 6763 raft_consensus.cc:348] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.740437 6763 raft_consensus.cc:374] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:23.740551 6763 raft_consensus.cc:729] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513b026fe15b45ca944708db85567088, State: Initialized, Role: FOLLOWER
I20260927 22:57:23.740842 6763 consensus_queue.cc:260] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.741463 6763 ts_tablet_manager.cc:1428] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260927 22:57:23.742048 6762 tablet_bootstrap.cc:492] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe: No bootstrap required, opened a new log
I20260927 22:57:23.742269 6762 ts_tablet_manager.cc:1397] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260927 22:57:23.743136 6762 raft_consensus.cc:348] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.743438 6762 raft_consensus.cc:374] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:23.743518 6762 raft_consensus.cc:729] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2313a7f01817402fbd824734bc8608fe, State: Initialized, Role: FOLLOWER
I20260927 22:57:23.743916 6762 consensus_queue.cc:260] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.744540 6762 ts_tablet_manager.cc:1428] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 22:57:23.958021 6768 raft_consensus.cc:482] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:23.958377 6768 raft_consensus.cc:504] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.959326 6768 leader_election.cc:288] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619), 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131:35309)
I20260927 22:57:23.960440 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2017c1f62cf4eac996414955aa09b3a" candidate_uuid: "513b026fe15b45ca944708db85567088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313a7f01817402fbd824734bc8608fe" is_pre_election: true
I20260927 22:57:23.960810 6452 raft_consensus.cc:2454] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513b026fe15b45ca944708db85567088 in term 0.
I20260927 22:57:23.960539 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2017c1f62cf4eac996414955aa09b3a" candidate_uuid: "513b026fe15b45ca944708db85567088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e819715daae426a9cc50cb06fef6b23" is_pre_election: true
I20260927 22:57:23.961544 6592 raft_consensus.cc:2454] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513b026fe15b45ca944708db85567088 in term 0.
I20260927 22:57:23.961685 6242 leader_election.cc:302] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313a7f01817402fbd824734bc8608fe, 513b026fe15b45ca944708db85567088; no voters:
I20260927 22:57:23.962098 6768 raft_consensus.cc:2781] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:23.962224 6768 raft_consensus.cc:482] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:23.962352 6768 raft_consensus.cc:3037] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:23.964908 6768 raft_consensus.cc:504] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.965569 6768 leader_election.cc:288] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [CANDIDATE]: Term 1 election: Requested vote from peers 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619), 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131:35309)
I20260927 22:57:23.966353 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2017c1f62cf4eac996414955aa09b3a" candidate_uuid: "513b026fe15b45ca944708db85567088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313a7f01817402fbd824734bc8608fe"
I20260927 22:57:23.966463 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2017c1f62cf4eac996414955aa09b3a" candidate_uuid: "513b026fe15b45ca944708db85567088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e819715daae426a9cc50cb06fef6b23"
I20260927 22:57:23.966638 6452 raft_consensus.cc:3037] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:23.966809 6592 raft_consensus.cc:3037] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:23.968988 6452 raft_consensus.cc:2454] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513b026fe15b45ca944708db85567088 in term 1.
I20260927 22:57:23.969403 6592 raft_consensus.cc:2454] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513b026fe15b45ca944708db85567088 in term 1.
I20260927 22:57:23.969628 6242 leader_election.cc:302] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313a7f01817402fbd824734bc8608fe, 513b026fe15b45ca944708db85567088; no voters:
I20260927 22:57:23.970233 6768 raft_consensus.cc:2781] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:23.970671 6768 raft_consensus.cc:686] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 1 LEADER]: Becoming Leader. State: Replica: 513b026fe15b45ca944708db85567088, State: Running, Role: LEADER
I20260927 22:57:23.971000 6768 consensus_queue.cc:237] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } }
I20260927 22:57:23.974092 6153 catalog_manager.cc:5495] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 reported cstate change: term changed from 0 to 1, leader changed from <none> to 513b026fe15b45ca944708db85567088 (127.4.74.129). New cstate: current_term: 1 leader_uuid: "513b026fe15b45ca944708db85567088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b026fe15b45ca944708db85567088" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 } health_report { overall_health: UNKNOWN } } }
I20260927 22:57:24.002362 6781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:23.995987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.4.74.129:34697)
0927 22:57:23.996597 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:23.996602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:23.996625 (+ 23us) server_negotiation.cc:407] Connection header received
0927 22:57:23.996688 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:23.996691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:23.996759 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:23.996932 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:23.997525 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:23.998398 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:24.000199 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:24.000623 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:24.000725 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:24.001298 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:24.001408 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:24.001484 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:24.001760 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 22:57:24.001832 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":353,"threads_started":1}
I20260927 22:57:24.014678 6592 raft_consensus.cc:1261] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Refusing update from remote peer 513b026fe15b45ca944708db85567088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:24.014922 6452 raft_consensus.cc:1261] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Refusing update from remote peer 513b026fe15b45ca944708db85567088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:24.015715 6779 consensus_queue.cc:1035] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2313a7f01817402fbd824734bc8608fe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:24.016147 6768 consensus_queue.cc:1035] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e819715daae426a9cc50cb06fef6b23" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 22:57:24.041420 6734 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 22:57:25.645323 6155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:25.645648 6155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:25.650735 6155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9dfff65842c34f488ff1623336e3eb5c: Sending DeleteTablet for 3 replicas of tablet d2017c1f62cf4eac996414955aa09b3a
I20260927 22:57:25.651835 6292 tablet_service.cc:1514] Processing DeleteTablet for tablet d2017c1f62cf4eac996414955aa09b3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:25 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:36346
I20260927 22:57:25.651924 6432 tablet_service.cc:1514] Processing DeleteTablet for tablet d2017c1f62cf4eac996414955aa09b3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:25 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:45788
I20260927 22:57:25.652141 6571 tablet_service.cc:1514] Processing DeleteTablet for tablet d2017c1f62cf4eac996414955aa09b3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:25 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:54936
I20260927 22:57:25.652980 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 6226
I20260927 22:57:25.667392 6809 tablet_replica.cc:330] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088: stopping tablet replica
I20260927 22:57:25.667762 6809 raft_consensus.cc:2229] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:57:25.668255 6809 raft_consensus.cc:2258] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:25.668972 6808 tablet_replica.cc:330] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe: stopping tablet replica
I20260927 22:57:25.669078 6810 tablet_replica.cc:330] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23: stopping tablet replica
I20260927 22:57:25.669198 6808 raft_consensus.cc:2229] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:25.669451 6808 raft_consensus.cc:2258] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:25.669467 6810 raft_consensus.cc:2229] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:25.669817 6810 raft_consensus.cc:2258] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:25.670504 6809 ts_tablet_manager.cc:1905] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:25.670949 6810 ts_tablet_manager.cc:1905] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:25.677939 6810 ts_tablet_manager.cc:1918] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 22:57:25.678009 6809 ts_tablet_manager.cc:1918] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 22:57:25.678133 6809 log.cc:1196] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/d2017c1f62cf4eac996414955aa09b3a
I20260927 22:57:25.678280 6810 log.cc:1196] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/d2017c1f62cf4eac996414955aa09b3a
I20260927 22:57:25.678642 6809 ts_tablet_manager.cc:1939] T d2017c1f62cf4eac996414955aa09b3a P 513b026fe15b45ca944708db85567088: Deleting consensus metadata
I20260927 22:57:25.678807 6810 ts_tablet_manager.cc:1939] T d2017c1f62cf4eac996414955aa09b3a P 3e819715daae426a9cc50cb06fef6b23: Deleting consensus metadata
I20260927 22:57:25.678776 6808 ts_tablet_manager.cc:1905] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:25.679960 6139 catalog_manager.cc:4841] TS 3e819715daae426a9cc50cb06fef6b23 (127.4.74.131:35309): tablet d2017c1f62cf4eac996414955aa09b3a (table test-table [id=fa583a4dcbbe4a8c8b138054ad444c90]) successfully deleted
I20260927 22:57:25.679960 6142 catalog_manager.cc:4841] TS 513b026fe15b45ca944708db85567088 (127.4.74.129:34697): tablet d2017c1f62cf4eac996414955aa09b3a (table test-table [id=fa583a4dcbbe4a8c8b138054ad444c90]) successfully deleted
Sep 27 22:57:25 dist-test-slave-5pd3 krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549843, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:25.687425 6808 ts_tablet_manager.cc:1918] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 22:57:25.687702 6808 log.cc:1196] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/d2017c1f62cf4eac996414955aa09b3a
I20260927 22:57:25.688275 6808 ts_tablet_manager.cc:1939] T d2017c1f62cf4eac996414955aa09b3a P 2313a7f01817402fbd824734bc8608fe: Deleting consensus metadata
I20260927 22:57:25.689713 6139 catalog_manager.cc:4841] TS 2313a7f01817402fbd824734bc8608fe (127.4.74.130:38619): tablet d2017c1f62cf4eac996414955aa09b3a (table test-table [id=fa583a4dcbbe4a8c8b138054ad444c90]) successfully deleted
I20260927 22:57:25.694283 6812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:25.655665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48514 (local address 127.4.74.129:34697)
0927 22:57:25.661343 (+ 5678us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:25.661354 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:25.668753 (+ 7399us) server_negotiation.cc:407] Connection header received
0927 22:57:25.669185 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:25.669191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:25.669307 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:25.669461 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:25.671634 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:25.673093 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:25.675462 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:25.676331 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:25.676468 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:25.687007 (+ 10539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:25.687025 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:25.687042 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:25.687085 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:25.691046 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:25.691968 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:25.691979 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:25.691985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:25.692134 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:25.692569 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:25.692574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:25.692579 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:25.692993 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:25.693276 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:25.693629 (+ 353us) server_negotiation.cc:299] Negotiation successful
0927 22:57:25.693888 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5440,"thread_start_us":5192,"threads_started":1}
W20260927 22:57:25.695114 6357 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48514
W20260927 22:57:25.695919 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 6226: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 22:57:25.696218 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 6226
I20260927 22:57:25.722424 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 6366
Sep 27 22:57:25 dist-test-slave-5pd3 krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549843, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:57:25.739926 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:25.723417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50574 (local address 127.4.74.130:38619)
0927 22:57:25.724097 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:25.724101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:25.724118 (+ 17us) server_negotiation.cc:407] Connection header received
0927 22:57:25.724200 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:25.724207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:25.724295 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:25.724435 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:25.725965 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:25.726701 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:25.728119 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:25.728539 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:25.728640 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:25.733445 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:25.733460 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:25.733476 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:25.733515 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:25.736565 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:25.737579 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:25.737586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:25.737589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:25.737777 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:25.738277 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:25.738280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:25.738282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:25.738641 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:25.738890 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:25.739372 (+ 482us) server_negotiation.cc:299] Negotiation successful
0927 22:57:25.739642 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":383,"threads_started":1}
W20260927 22:57:25.740756 6497 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50574
W20260927 22:57:25.741631 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 6366: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 22:57:25.741854 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 6366
I20260927 22:57:25.760471 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 6506
Sep 27 22:57:25 dist-test-slave-5pd3 krb5kdc[6110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549843, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:57:25.778188 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:25.761628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.4.74.131:35309)
0927 22:57:25.762263 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:25.762272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:25.762293 (+ 21us) server_negotiation.cc:407] Connection header received
0927 22:57:25.762367 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:25.762372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:25.762448 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:25.762584 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:25.764406 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:25.765251 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:25.766555 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:25.767003 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:25.767147 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:25.772192 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:25.772211 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:25.772228 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:25.772282 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:25.775198 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:25.776120 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:25.776127 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:25.776129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:25.776264 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:25.776693 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:25.776696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:25.776698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:25.777067 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:25.777298 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:25.777648 (+ 350us) server_negotiation.cc:299] Negotiation successful
0927 22:57:25.777893 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":275,"threads_started":1}
W20260927 22:57:25.778992 6637 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45568
W20260927 22:57:25.779841 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 6506: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 22:57:25.780061 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 6506
I20260927 22:57:25.805624 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 6123
I20260927 22:57:25.806972 6195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:25.947099 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 6123
2026-09-27T22:57:25Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9484 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:26 dist-test-slave-5pd3 krb5kdc[6818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:26 dist-test-slave-5pd3 krb5kdc[6818](info): set up 2 sockets
Sep 27 22:57:26 dist-test-slave-5pd3 krb5kdc[6818](info): commencing operation
krb5kdc: starting...
W20260927 22:57:27.060320 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.006s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:27 dist-test-slave-5pd3 krb5kdc[6818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:57:27.287695 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:35649
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:43729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:35649
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:27.625583 6831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:27.625914 6831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:27.626075 6831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:27.633760 6831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:27.633896 6831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:27.633945 6831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:27.634078 6831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:27.634147 6831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:27.644608 6831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:43729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:35649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:35649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:27.646060 6831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:27.647415 6831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:27.666486 6836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:27.666592 6837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:27.666554 6839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:27.668064 6831 server_base.cc:1034] running on GCE node
I20260927 22:57:27.669329 6831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:27.671307 6831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:27.672591 6831 hybrid_clock.cc:648] HybridClock initialized: now 1790549847672529 us; error 59 us; skew 500 ppm
I20260927 22:57:27.673054 6831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:27 dist-test-slave-5pd3 krb5kdc[6818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549847, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:27.676851 6831 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:57:27.678767 6831 webserver.cc:460] Webserver started at http://127.4.74.190:34471/ using document root <none> and password file <none>
I20260927 22:57:27.679805 6831 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:27.679966 6831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:27.680351 6831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:27.684087 6831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "e0ab2ffc52e74c20bb3f6257a0619a53"
format_stamp: "Formatted at 2026-09-27 22:57:27 on dist-test-slave-5pd3"
server_key: "93bfab3fa8919a4ce064ce4c4761a65e"
server_key_iv: "72091402ac872329db6e4bace8c0908c"
server_key_version: "encryptionkey@0"
I20260927 22:57:27.685043 6831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "e0ab2ffc52e74c20bb3f6257a0619a53"
format_stamp: "Formatted at 2026-09-27 22:57:27 on dist-test-slave-5pd3"
server_key: "93bfab3fa8919a4ce064ce4c4761a65e"
server_key_iv: "72091402ac872329db6e4bace8c0908c"
server_key_version: "encryptionkey@0"
I20260927 22:57:27.691725 6831 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260927 22:57:27.696195 6846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:27.698024 6831 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:57:27.698320 6831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "e0ab2ffc52e74c20bb3f6257a0619a53"
format_stamp: "Formatted at 2026-09-27 22:57:27 on dist-test-slave-5pd3"
server_key: "93bfab3fa8919a4ce064ce4c4761a65e"
server_key_iv: "72091402ac872329db6e4bace8c0908c"
server_key_version: "encryptionkey@0"
I20260927 22:57:27.698587 6831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:27.738452 6831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:27.744355 6831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:27.744658 6831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:27.767927 6831 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:35649
I20260927 22:57:27.767963 6897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:35649 every 8 connection(s)
I20260927 22:57:27.769876 6831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:27.771535 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 6831
I20260927 22:57:27.771818 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:27.772620 4394 external_mini_cluster.cc:1428] Setting key b995811582bbb066ca4ee4666d4b8c74
I20260927 22:57:27.775761 6898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:27.789960 6898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53: Bootstrap starting.
Sep 27 22:57:27 dist-test-slave-5pd3 krb5kdc[6818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:27.793442 6898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:27.794700 6898 log.cc:824] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:27.798758 6898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53: No bootstrap required, opened a new log
I20260927 22:57:27.802676 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:27.775260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35426 (local address 127.4.74.190:35649)
0927 22:57:27.776339 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:27.776354 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:27.776408 (+ 54us) server_negotiation.cc:407] Connection header received
0927 22:57:27.778301 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:27.778328 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:27.778859 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:27.779313 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:27.781324 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:27.783122 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:27.785107 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:27.786107 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:27.786311 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:27.791770 (+ 5459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:27.791780 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:27.791801 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:27.791857 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:27.796589 (+ 4732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:27.797688 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:27.797697 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:27.797706 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:27.797949 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:27.798556 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:27.798562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:27.798566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:27.799487 (+ 921us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:27.799877 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:27.800492 (+ 615us) server_negotiation.cc:299] Negotiation successful
0927 22:57:27.801114 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":531,"threads_started":1}
I20260927 22:57:27.808475 6898 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ab2ffc52e74c20bb3f6257a0619a53" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35649 } }
I20260927 22:57:27.808934 6898 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:27.809056 6898 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0ab2ffc52e74c20bb3f6257a0619a53, State: Initialized, Role: FOLLOWER
I20260927 22:57:27.809628 6898 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ab2ffc52e74c20bb3f6257a0619a53" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35649 } }
I20260927 22:57:27.810149 6898 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:27.810319 6898 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:27.810647 6898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:27.813194 6898 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ab2ffc52e74c20bb3f6257a0619a53" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35649 } }
I20260927 22:57:27.813863 6898 leader_election.cc:302] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0ab2ffc52e74c20bb3f6257a0619a53; no voters:
I20260927 22:57:27.814538 6898 leader_election.cc:288] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:27.814780 6903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:27.815693 6903 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [term 1 LEADER]: Becoming Leader. State: Replica: e0ab2ffc52e74c20bb3f6257a0619a53, State: Running, Role: LEADER
I20260927 22:57:27.816406 6903 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ab2ffc52e74c20bb3f6257a0619a53" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35649 } }
I20260927 22:57:27.817056 6898 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:27.820148 6904 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0ab2ffc52e74c20bb3f6257a0619a53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ab2ffc52e74c20bb3f6257a0619a53" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35649 } } }
I20260927 22:57:27.820725 6904 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:27.821619 6905 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0ab2ffc52e74c20bb3f6257a0619a53. Latest consensus state: current_term: 1 leader_uuid: "e0ab2ffc52e74c20bb3f6257a0619a53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ab2ffc52e74c20bb3f6257a0619a53" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35649 } } }
I20260927 22:57:27.822028 6905 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:27.823535 6908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:27.830855 6908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:27.842856 6908 catalog_manager.cc:1347] Generated new cluster ID: 35eae118e7184d0aa4cc161aa90cb7f3
I20260927 22:57:27.843020 6908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:27.890228 6908 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:27.892179 6908 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:27.905421 6908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0ab2ffc52e74c20bb3f6257a0619a53: Generated new TSK 0
I20260927 22:57:27.906929 6908 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:57:27.969645 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35649
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:43729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:28.332094 6924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:28.332545 6924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:28.332701 6924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:28.340344 6924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:28.340480 6924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:28.340663 6924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:57:28.351029 6924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:43729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:28.352705 6924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:28.354183 6924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:28.371842 6930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:28.371825 6929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:28.372104 6932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:28.372797 6924 server_base.cc:1034] running on GCE node
I20260927 22:57:28.373735 6924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:28.375131 6924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:28.376446 6924 hybrid_clock.cc:648] HybridClock initialized: now 1790549848376415 us; error 154 us; skew 500 ppm
I20260927 22:57:28.376920 6924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:28 dist-test-slave-5pd3 krb5kdc[6818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549848, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:28.381136 6924 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:57:28.383172 6924 webserver.cc:460] Webserver started at http://127.4.74.129:41037/ using document root <none> and password file <none>
I20260927 22:57:28.384347 6924 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:28.384541 6924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:28.384948 6924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:28.388935 6924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "e1699a3366aa40149da9c1c6d7bff5be"
format_stamp: "Formatted at 2026-09-27 22:57:28 on dist-test-slave-5pd3"
server_key: "d89e51a46cc01055b54c134ffe0ab90f"
server_key_iv: "d8a1e7fa868a8efc32a4bad03c4dc091"
server_key_version: "encryptionkey@0"
I20260927 22:57:28.389984 6924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "e1699a3366aa40149da9c1c6d7bff5be"
format_stamp: "Formatted at 2026-09-27 22:57:28 on dist-test-slave-5pd3"
server_key: "d89e51a46cc01055b54c134ffe0ab90f"
server_key_iv: "d8a1e7fa868a8efc32a4bad03c4dc091"
server_key_version: "encryptionkey@0"
I20260927 22:57:28.397245 6924 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260927 22:57:28.402489 6939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:28.404773 6924 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260927 22:57:28.405092 6924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "e1699a3366aa40149da9c1c6d7bff5be"
format_stamp: "Formatted at 2026-09-27 22:57:28 on dist-test-slave-5pd3"
server_key: "d89e51a46cc01055b54c134ffe0ab90f"
server_key_iv: "d8a1e7fa868a8efc32a4bad03c4dc091"
server_key_version: "encryptionkey@0"
I20260927 22:57:28.405391 6924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:28.420295 6924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:28.425940 6924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:28.426370 6924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:28.427693 6924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:28.429411 6924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:28.429531 6924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:28.429679 6924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:28.429757 6924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:28.470793 6924 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:36159
I20260927 22:57:28.470901 7051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:36159 every 8 connection(s)
I20260927 22:57:28.472340 6924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:57:28.478088 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 6924
I20260927 22:57:28.478359 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:57:28.479063 4394 external_mini_cluster.cc:1428] Setting key f2b47b8e46ea3a7f9f663965d4209325
Sep 27 22:57:28 dist-test-slave-5pd3 krb5kdc[6818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549848, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:28.499630 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:28.475712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:49343 (local address 127.4.74.190:35649)
0927 22:57:28.476260 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:28.476265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:28.477547 (+ 1282us) server_negotiation.cc:407] Connection header received
0927 22:57:28.479980 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:28.479987 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:28.480152 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:28.480332 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:28.482505 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:28.483953 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:28.485833 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:28.486659 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:28.486853 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:28.491684 (+ 4831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:28.491690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:28.491695 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:28.491753 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:28.494857 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:28.496106 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:28.496118 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:28.496125 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:28.496381 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:28.497068 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:28.497075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:28.497081 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:28.497549 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:28.497753 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:28.498772 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0927 22:57:28.499099 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":304,"threads_started":1}
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 22:57:28.501116 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:28.476417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35649 (local address 127.4.74.129:49343)
0927 22:57:28.477344 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:28.477396 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:28.479509 (+ 2113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:28.480696 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:28.480710 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:28.481571 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:28.482266 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:28.482285 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:28.484189 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:28.484193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:28.485616 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:28.485635 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:28.486963 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:28.486969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:28.487185 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:28.488117 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:28.488144 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:28.491262 (+ 3118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:28.495102 (+ 3840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:28.495116 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:28.495146 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:28.495847 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:28.496613 (+ 766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:28.496617 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:28.496620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:28.496890 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:28.497731 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:28.497740 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:28.498233 (+ 493us) client_negotiation.cc:769] Sending connection context
0927 22:57:28.498557 (+ 324us) client_negotiation.cc:240] Negotiation successful
0927 22:57:28.499319 (+ 762us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":367,"threads_started":1}
I20260927 22:57:28.503289 7052 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
I20260927 22:57:28.503911 7052 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:28.504943 7052 heartbeater.cc:511] Master 127.4.74.190:35649 requested a full tablet report, sending...
I20260927 22:57:28.508013 6863 ts_manager.cc:194] Registered new tserver with Master: e1699a3366aa40149da9c1c6d7bff5be (127.4.74.129:36159)
I20260927 22:57:28.510423 6863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:49343
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:57:28.532747 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35649
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:43729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:28.885865 7058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:28.886195 7058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:28.886356 7058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:28.895272 7058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:28.895413 7058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:28.895622 7058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:57:28.907353 7058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:43729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:28.909265 7058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:28.911063 7058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:28.930648 7063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:28.930585 7064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:28.930612 7066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:28.931325 7058 server_base.cc:1034] running on GCE node
I20260927 22:57:28.932109 7058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:28.933305 7058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:28.934664 7058 hybrid_clock.cc:648] HybridClock initialized: now 1790549848934600 us; error 67 us; skew 500 ppm
I20260927 22:57:28.935155 7058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:28 dist-test-slave-5pd3 krb5kdc[6818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549848, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:28.942457 7058 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:57:28.945932 7058 webserver.cc:460] Webserver started at http://127.4.74.130:36155/ using document root <none> and password file <none>
I20260927 22:57:28.947093 7058 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:28.947412 7058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:28.947865 7058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:28.952987 7058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "e1ae538d70854e148353f182b56c5ddd"
format_stamp: "Formatted at 2026-09-27 22:57:28 on dist-test-slave-5pd3"
server_key: "68f073f7d8fa21d8355e849b22bd9b20"
server_key_iv: "d54f0d9f71c6eaa20a47f6929187f85d"
server_key_version: "encryptionkey@0"
I20260927 22:57:28.954578 7058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "e1ae538d70854e148353f182b56c5ddd"
format_stamp: "Formatted at 2026-09-27 22:57:28 on dist-test-slave-5pd3"
server_key: "68f073f7d8fa21d8355e849b22bd9b20"
server_key_iv: "d54f0d9f71c6eaa20a47f6929187f85d"
server_key_version: "encryptionkey@0"
I20260927 22:57:28.963610 7058 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s
I20260927 22:57:28.970677 7073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:28.973102 7058 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260927 22:57:28.973392 7058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "e1ae538d70854e148353f182b56c5ddd"
format_stamp: "Formatted at 2026-09-27 22:57:28 on dist-test-slave-5pd3"
server_key: "68f073f7d8fa21d8355e849b22bd9b20"
server_key_iv: "d54f0d9f71c6eaa20a47f6929187f85d"
server_key_version: "encryptionkey@0"
I20260927 22:57:28.973658 7058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:29.005239 7058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:29.010646 7058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:29.011026 7058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:29.012310 7058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:29.014007 7058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:29.014096 7058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:29.014241 7058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:29.014307 7058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:29.056270 7058 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:44039
I20260927 22:57:29.056437 7185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:44039 every 8 connection(s)
I20260927 22:57:29.057657 7058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:57:29.066258 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 7058
I20260927 22:57:29.066555 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:57:29.067191 4394 external_mini_cluster.cc:1428] Setting key 42da59ddf2d00bf21f74aeb10897b10a
Sep 27 22:57:29 dist-test-slave-5pd3 krb5kdc[6818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549848, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:29.084407 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:29.061163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:36611 (local address 127.4.74.190:35649)
0927 22:57:29.061813 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:29.061819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:29.062659 (+ 840us) server_negotiation.cc:407] Connection header received
0927 22:57:29.064733 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:29.064740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:29.064849 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:29.064997 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:29.066929 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:29.068070 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:29.070449 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:29.071212 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:29.071387 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:29.076428 (+ 5041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:29.076438 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:29.076446 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:29.076520 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:29.079671 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:29.081230 (+ 1559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:29.081237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:29.081239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:29.081437 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:29.082117 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:29.082122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:29.082124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:29.082483 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:29.082649 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:29.083795 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0927 22:57:29.084071 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":373,"threads_started":1}
I20260927 22:57:29.086329 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:29.061490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35649 (local address 127.4.74.130:36611)
0927 22:57:29.062396 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:29.062472 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:29.064407 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:29.065282 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:29.065304 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:29.065891 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:29.066678 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:29.066706 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:29.068317 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:29.068323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:29.070254 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:29.070268 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:29.071518 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:29.071523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:29.071848 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:29.072630 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:29.072655 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:29.076044 (+ 3389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:29.080006 (+ 3962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:29.080021 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:29.080043 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:29.080833 (+ 790us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:29.081607 (+ 774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:29.081615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:29.081622 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:29.081919 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:29.082658 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:29.082670 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:29.083199 (+ 529us) client_negotiation.cc:769] Sending connection context
0927 22:57:29.083658 (+ 459us) client_negotiation.cc:240] Negotiation successful
0927 22:57:29.084523 (+ 865us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":397,"threads_started":1}
I20260927 22:57:29.088511 7186 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35649
I20260927 22:57:29.088999 7186 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 22:57:29.090065 7186 heartbeater.cc:511] Master 127.4.74.190:35649 requested a full tablet report, sending...
I20260927 22:57:29.092393 6863 ts_manager.cc:194] Registered new tserver with Master: e1ae538d70854e148353f182b56c5ddd (127.4.74.130:44039)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
I20260927 22:57:29.093865 6863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:36611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:57:29.123413 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35649
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:43729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:29.483134 7192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:29.483505 7192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:29.483752 7192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:29.491163 7192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:29.491290 7192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:29.491405 7192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:57:29.502056 7192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:43729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:29.503600 7192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:29.505025 7192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 22:57:29.514485 7052 heartbeater.cc:503] Master 127.4.74.190:35649 was elected leader, sending a full tablet report...
W20260927 22:57:29.520879 7198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:29.521137 7200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:29.521198 7197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:29.522675 7192 server_base.cc:1034] running on GCE node
I20260927 22:57:29.523423 7192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:29.524709 7192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:29.526055 7192 hybrid_clock.cc:648] HybridClock initialized: now 1790549849525975 us; error 67 us; skew 500 ppm
I20260927 22:57:29.526499 7192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:29 dist-test-slave-5pd3 krb5kdc[6818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549849, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:29.530248 7192 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:57:29.532203 7192 webserver.cc:460] Webserver started at http://127.4.74.131:43127/ using document root <none> and password file <none>
I20260927 22:57:29.532963 7192 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:29.533115 7192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:29.533449 7192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:29.536662 7192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "eed151afdaf249bbbef80d803a37a05c"
format_stamp: "Formatted at 2026-09-27 22:57:29 on dist-test-slave-5pd3"
server_key: "37657aab62f2eef8a061d6a792a65eb3"
server_key_iv: "9f39eb2cd1f2a18988b835c0d4518075"
server_key_version: "encryptionkey@0"
I20260927 22:57:29.537519 7192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "eed151afdaf249bbbef80d803a37a05c"
format_stamp: "Formatted at 2026-09-27 22:57:29 on dist-test-slave-5pd3"
server_key: "37657aab62f2eef8a061d6a792a65eb3"
server_key_iv: "9f39eb2cd1f2a18988b835c0d4518075"
server_key_version: "encryptionkey@0"
I20260927 22:57:29.543026 7192 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260927 22:57:29.547196 7207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:29.549147 7192 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:57:29.549391 7192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "eed151afdaf249bbbef80d803a37a05c"
format_stamp: "Formatted at 2026-09-27 22:57:29 on dist-test-slave-5pd3"
server_key: "37657aab62f2eef8a061d6a792a65eb3"
server_key_iv: "9f39eb2cd1f2a18988b835c0d4518075"
server_key_version: "encryptionkey@0"
I20260927 22:57:29.549628 7192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:29.576427 7192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:29.582109 7192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:29.582484 7192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:29.583757 7192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:29.585351 7192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:29.585475 7192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:29.585636 7192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:29.585711 7192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:29.630367 7192 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:35719
I20260927 22:57:29.630431 7319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:35719 every 8 connection(s)
I20260927 22:57:29.632272 7192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:57:29.639150 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 7192
I20260927 22:57:29.639355 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:57:29.639891 4394 external_mini_cluster.cc:1428] Setting key 1d4f508148d8c4d28a4bfc8db88c7499
Sep 27 22:57:29 dist-test-slave-5pd3 krb5kdc[6818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549849, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:29.660459 7322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:29.635981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:44747 (local address 127.4.74.190:35649)
0927 22:57:29.636559 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:29.636563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:29.638010 (+ 1447us) server_negotiation.cc:407] Connection header received
0927 22:57:29.640574 (+ 2564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:29.640580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:29.640689 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:29.640821 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:29.642670 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:29.643640 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:29.646140 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:29.646604 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:29.646747 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:29.651857 (+ 5110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:29.651863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:29.651868 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:29.651926 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:29.655455 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:29.656603 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:29.656644 (+ 41us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:29.656650 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:29.656935 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:29.657803 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:29.657810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:29.657813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:29.658391 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:29.658554 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:29.659443 (+ 889us) server_negotiation.cc:299] Negotiation successful
0927 22:57:29.659810 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":325,"threads_started":1}
I20260927 22:57:29.661293 7323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:29.636522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35649 (local address 127.4.74.131:44747)
0927 22:57:29.637648 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:29.637708 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:29.640227 (+ 2519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:29.641134 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:29.641148 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:29.641851 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:29.642468 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:29.642486 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:29.643862 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:29.643868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:29.645734 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:29.645759 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:29.646948 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:29.646955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:29.647245 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:29.648230 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:29.648260 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:29.651432 (+ 3172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:29.655685 (+ 4253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:29.655697 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:29.655718 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:29.656421 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:29.657127 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:29.657132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:29.657136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:29.657540 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:29.658565 (+ 1025us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:29.658572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:29.659014 (+ 442us) client_negotiation.cc:769] Sending connection context
0927 22:57:29.659391 (+ 377us) client_negotiation.cc:240] Negotiation successful
0927 22:57:29.660060 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":412,"threads_started":1}
I20260927 22:57:29.663365 7320 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35649
I20260927 22:57:29.663915 7320 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:29.664954 7320 heartbeater.cc:511] Master 127.4.74.190:35649 requested a full tablet report, sending...
I20260927 22:57:29.667990 6863 ts_manager.cc:194] Registered new tserver with Master: eed151afdaf249bbbef80d803a37a05c (127.4.74.131:35719)
I20260927 22:57:29.669003 6863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:44747
I20260927 22:57:29.672036 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 22:57:29.712378 7322 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:57:29.709066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35440 (local address 127.4.74.190:35649)
0927 22:57:29.709338 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:29.709354 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:29.709940 (+ 586us) server_negotiation.cc:407] Connection header received
0927 22:57:29.710230 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:29.710239 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:29.710359 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:29.710483 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:29.712123 (+ 1640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35440: BlockingRecv error: recv got EOF from 127.0.0.1:35440 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 22:57:29.715801 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 6924
W20260927 22:57:29.719240 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 6924: Not authorized: Client connection negotiation failed: client connection to 127.4.74.129:36159: server requires authentication, but client does not have Kerberos credentials available
I20260927 22:57:29.719488 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 6924
W20260927 22:57:29.719647 7332 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:57:29.717063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43000 (local address 127.4.74.129:36159)
0927 22:57:29.717747 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:29.717754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:29.717784 (+ 30us) server_negotiation.cc:407] Connection header received
0927 22:57:29.717866 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:29.717873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:29.718149 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:29.718329 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:29.719224 (+ 895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43000: BlockingRecv error: recv got EOF from 127.0.0.1:43000 (error 108)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":347,"threads_started":1}
I20260927 22:57:29.737468 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7058
W20260927 22:57:29.741122 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7058: Not authorized: Client connection negotiation failed: client connection to 127.4.74.130:44039: server requires authentication, but client does not have Kerberos credentials available
I20260927 22:57:29.741406 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 7058
I20260927 22:57:29.759322 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7192
W20260927 22:57:29.763789 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7192: Not authorized: Client connection negotiation failed: client connection to 127.4.74.131:35719: server requires authentication, but client does not have Kerberos credentials available
I20260927 22:57:29.763967 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 7192
I20260927 22:57:29.787710 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 6831
I20260927 22:57:29.789510 6893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:29.929920 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 6831
2026-09-27T22:57:29Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3993 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:30 dist-test-slave-5pd3 krb5kdc[7338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:30 dist-test-slave-5pd3 krb5kdc[7338](info): set up 2 sockets
Sep 27 22:57:30 dist-test-slave-5pd3 krb5kdc[7338](info): commencing operation
krb5kdc: starting...
W20260927 22:57:31.104933 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:31 dist-test-slave-5pd3 krb5kdc[7338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:57:31.341348 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:42629
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:42629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:31.689908 7351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:31.690236 7351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:31.690402 7351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:31.698010 7351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:31.698148 7351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:31.698194 7351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:31.698283 7351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:31.698371 7351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:31.708753 7351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:42629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:31.710204 7351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:31.711745 7351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:31.725577 7356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:31.725809 7357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:31.726410 7359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:31.727041 7351 server_base.cc:1034] running on GCE node
I20260927 22:57:31.728196 7351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:31.730185 7351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:31.731482 7351 hybrid_clock.cc:648] HybridClock initialized: now 1790549851731410 us; error 71 us; skew 500 ppm
I20260927 22:57:31.731982 7351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:31 dist-test-slave-5pd3 krb5kdc[7338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549851, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:31.741205 7351 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:57:31.743702 7351 webserver.cc:460] Webserver started at http://127.4.74.190:35475/ using document root <none> and password file <none>
I20260927 22:57:31.744688 7351 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:31.744863 7351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:31.745256 7351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:31.749109 7351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "69921ca4452b464dab5e0f0dea15fe2c"
format_stamp: "Formatted at 2026-09-27 22:57:31 on dist-test-slave-5pd3"
server_key: "2041512b2a977ea442d13f9d790200df"
server_key_iv: "c326f492582b59acfc6fa09fa6759b98"
server_key_version: "encryptionkey@0"
I20260927 22:57:31.750111 7351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "69921ca4452b464dab5e0f0dea15fe2c"
format_stamp: "Formatted at 2026-09-27 22:57:31 on dist-test-slave-5pd3"
server_key: "2041512b2a977ea442d13f9d790200df"
server_key_iv: "c326f492582b59acfc6fa09fa6759b98"
server_key_version: "encryptionkey@0"
I20260927 22:57:31.757115 7351 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260927 22:57:31.762262 7366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:31.764585 7351 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260927 22:57:31.764909 7351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "69921ca4452b464dab5e0f0dea15fe2c"
format_stamp: "Formatted at 2026-09-27 22:57:31 on dist-test-slave-5pd3"
server_key: "2041512b2a977ea442d13f9d790200df"
server_key_iv: "c326f492582b59acfc6fa09fa6759b98"
server_key_version: "encryptionkey@0"
I20260927 22:57:31.765200 7351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:31.784626 7351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:31.790970 7351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:31.791404 7351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:31.816082 7351 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:42629
I20260927 22:57:31.816118 7417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:42629 every 8 connection(s)
I20260927 22:57:31.817991 7351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:31.822669 7418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:31.825495 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 7351
I20260927 22:57:31.825801 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:31.827059 4394 external_mini_cluster.cc:1428] Setting key 0a6b7b0100bd548e68fb15b753282af5
I20260927 22:57:31.839274 7418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c: Bootstrap starting.
I20260927 22:57:31.843791 7418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:31.845268 7418 log.cc:824] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c: Log is configured to *not* fsync() on all Append() calls
Sep 27 22:57:31 dist-test-slave-5pd3 krb5kdc[7338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:31.850685 7418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c: No bootstrap required, opened a new log
I20260927 22:57:31.861923 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:31.830132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.4.74.190:42629)
0927 22:57:31.831312 (+ 1180us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:31.831331 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:31.831397 (+ 66us) server_negotiation.cc:407] Connection header received
0927 22:57:31.833843 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:31.833890 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:31.834760 (+ 870us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:31.835376 (+ 616us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:31.837385 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:31.839827 (+ 2442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:31.841706 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:31.843013 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:31.843289 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:31.849656 (+ 6367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:31.849674 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:31.849702 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:31.849766 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:31.854996 (+ 5230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:31.856270 (+ 1274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:31.856294 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:31.856308 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:31.856603 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:31.857211 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:31.857217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:31.857221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:31.858267 (+ 1046us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:31.858668 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:31.859206 (+ 538us) server_negotiation.cc:299] Negotiation successful
0927 22:57:31.859917 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":533,"threads_started":1}
I20260927 22:57:31.863222 7418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69921ca4452b464dab5e0f0dea15fe2c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42629 } }
I20260927 22:57:31.863829 7418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:31.863961 7418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69921ca4452b464dab5e0f0dea15fe2c, State: Initialized, Role: FOLLOWER
I20260927 22:57:31.864820 7418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69921ca4452b464dab5e0f0dea15fe2c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42629 } }
I20260927 22:57:31.865154 7418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:31.865314 7418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:31.865535 7418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:31.868054 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 22:57:31.869393 7418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69921ca4452b464dab5e0f0dea15fe2c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42629 } }
I20260927 22:57:31.870335 7418 leader_election.cc:302] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69921ca4452b464dab5e0f0dea15fe2c; no voters:
I20260927 22:57:31.871404 7418 leader_election.cc:288] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:31.871589 7423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:31.873006 7423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [term 1 LEADER]: Becoming Leader. State: Replica: 69921ca4452b464dab5e0f0dea15fe2c, State: Running, Role: LEADER
I20260927 22:57:31.873713 7423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69921ca4452b464dab5e0f0dea15fe2c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42629 } }
I20260927 22:57:31.874529 7418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:31.877003 7425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69921ca4452b464dab5e0f0dea15fe2c. Latest consensus state: current_term: 1 leader_uuid: "69921ca4452b464dab5e0f0dea15fe2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69921ca4452b464dab5e0f0dea15fe2c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42629 } } }
I20260927 22:57:31.877467 7425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:31.878085 7424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69921ca4452b464dab5e0f0dea15fe2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69921ca4452b464dab5e0f0dea15fe2c" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42629 } } }
I20260927 22:57:31.878402 7424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:31.902029 7428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:31.916112 7428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:31.939436 7428 catalog_manager.cc:1347] Generated new cluster ID: b6569df3ec67459e9477014492249c03
I20260927 22:57:31.939674 7428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:31.969626 7446 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 22:57:31.971992 7428 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:31.974299 7428 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:31.984486 7428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69921ca4452b464dab5e0f0dea15fe2c: Generated new TSK 0
I20260927 22:57:31.985774 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:31.973816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49300 (local address 127.4.74.190:42629)
0927 22:57:31.974084 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:31.974092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:31.975298 (+ 1206us) server_negotiation.cc:407] Connection header received
0927 22:57:31.975521 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:31.975529 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:31.975675 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:31.975835 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:31.977329 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:31.978529 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:31.980696 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:31.981415 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:31.981576 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:31.983208 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:31.983217 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:31.983224 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:31.984300 (+ 1076us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:31.984422 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:31.984479 (+ 57us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:31.984638 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:31.984983 (+ 345us) server_negotiation.cc:299] Negotiation successful
0927 22:57:31.985316 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260927 22:57:31.985958 7428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 22:57:32.000875 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7351
I20260927 22:57:32.002967 7413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:32.098872 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 7351
2026-09-27T22:57:32Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2152 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:32 dist-test-slave-5pd3 krb5kdc[7453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:32 dist-test-slave-5pd3 krb5kdc[7453](info): set up 2 sockets
Sep 27 22:57:32 dist-test-slave-5pd3 krb5kdc[7453](info): commencing operation
krb5kdc: starting...
W20260927 22:57:33.288322 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:33 dist-test-slave-5pd3 krb5kdc[7453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:57:33.515774 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:34919
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:34919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:33.857163 7466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:33.857493 7466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:33.857661 7466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:33.865151 7466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:33.865298 7466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:33.865348 7466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:33.865525 7466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:33.865670 7466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:33.875949 7466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:34919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:34919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:33.877405 7466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:33.878720 7466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:33.892371 7471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:33.892500 7472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:33.892210 7474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:33.894213 7466 server_base.cc:1034] running on GCE node
I20260927 22:57:33.895334 7466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:33.897307 7466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:33.898577 7466 hybrid_clock.cc:648] HybridClock initialized: now 1790549853898500 us; error 50 us; skew 500 ppm
I20260927 22:57:33.898979 7466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:33 dist-test-slave-5pd3 krb5kdc[7453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549853, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:33.903770 7466 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:57:33.906109 7466 webserver.cc:460] Webserver started at http://127.4.74.190:40915/ using document root <none> and password file <none>
I20260927 22:57:33.907186 7466 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:33.907368 7466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:33.907823 7466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:33.912159 7466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "2795b79226bc469ca71b9f63a1f5db8f"
format_stamp: "Formatted at 2026-09-27 22:57:33 on dist-test-slave-5pd3"
server_key: "e7ebea05fe9d927976385f343644df1a"
server_key_iv: "1c0ec03ca6b4b6a34eca79ddb787668f"
server_key_version: "encryptionkey@0"
I20260927 22:57:33.913219 7466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "2795b79226bc469ca71b9f63a1f5db8f"
format_stamp: "Formatted at 2026-09-27 22:57:33 on dist-test-slave-5pd3"
server_key: "e7ebea05fe9d927976385f343644df1a"
server_key_iv: "1c0ec03ca6b4b6a34eca79ddb787668f"
server_key_version: "encryptionkey@0"
I20260927 22:57:33.921432 7466 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260927 22:57:33.926915 7481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:33.929127 7466 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:57:33.929448 7466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "2795b79226bc469ca71b9f63a1f5db8f"
format_stamp: "Formatted at 2026-09-27 22:57:33 on dist-test-slave-5pd3"
server_key: "e7ebea05fe9d927976385f343644df1a"
server_key_iv: "1c0ec03ca6b4b6a34eca79ddb787668f"
server_key_version: "encryptionkey@0"
I20260927 22:57:33.929740 7466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:33.962834 7466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:33.968485 7466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:33.968819 7466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:33.991508 7466 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:34919
I20260927 22:57:33.991621 7532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:34919 every 8 connection(s)
I20260927 22:57:33.993320 7466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:33.997565 7533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:34.001417 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 7466
I20260927 22:57:34.001652 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:34.002503 4394 external_mini_cluster.cc:1428] Setting key cdc1c02fd4b7b8535c12751e1c6ef530
I20260927 22:57:34.010432 7533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f: Bootstrap starting.
I20260927 22:57:34.015112 7533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:34.016691 7533 log.cc:824] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:34.020623 7533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f: No bootstrap required, opened a new log
Sep 27 22:57:34 dist-test-slave-5pd3 krb5kdc[7453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:34.029368 7533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2795b79226bc469ca71b9f63a1f5db8f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34919 } }
I20260927 22:57:34.029839 7533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:34.029991 7533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2795b79226bc469ca71b9f63a1f5db8f, State: Initialized, Role: FOLLOWER
I20260927 22:57:34.030722 7533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2795b79226bc469ca71b9f63a1f5db8f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34919 } }
I20260927 22:57:34.031021 7533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:34.031219 7533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:34.031437 7533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:34.033901 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:34.005230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48946 (local address 127.4.74.190:34919)
0927 22:57:34.006204 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:34.006219 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:34.006273 (+ 54us) server_negotiation.cc:407] Connection header received
0927 22:57:34.009600 (+ 3327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:34.009640 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:34.010305 (+ 665us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:34.010870 (+ 565us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:34.012788 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:34.014418 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:34.016977 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:34.018456 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:34.018732 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:34.023703 (+ 4971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:34.023716 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:34.023749 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:34.023811 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:34.028219 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:34.029242 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:34.029251 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:34.029261 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:34.029472 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:34.030011 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:34.030019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:34.030024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:34.030878 (+ 854us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:34.031275 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:34.031851 (+ 576us) server_negotiation.cc:299] Negotiation successful
0927 22:57:34.032398 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":453,"threads_started":1}
I20260927 22:57:34.035012 7533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2795b79226bc469ca71b9f63a1f5db8f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34919 } }
I20260927 22:57:34.035856 7533 leader_election.cc:302] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2795b79226bc469ca71b9f63a1f5db8f; no voters:
I20260927 22:57:34.036701 7533 leader_election.cc:288] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:34.036815 7538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:34.038293 7538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [term 1 LEADER]: Becoming Leader. State: Replica: 2795b79226bc469ca71b9f63a1f5db8f, State: Running, Role: LEADER
I20260927 22:57:34.039458 7533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:34.039585 7538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2795b79226bc469ca71b9f63a1f5db8f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34919 } }
I20260927 22:57:34.043495 7539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2795b79226bc469ca71b9f63a1f5db8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2795b79226bc469ca71b9f63a1f5db8f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34919 } } }
I20260927 22:57:34.043891 7540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2795b79226bc469ca71b9f63a1f5db8f. Latest consensus state: current_term: 1 leader_uuid: "2795b79226bc469ca71b9f63a1f5db8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2795b79226bc469ca71b9f63a1f5db8f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34919 } } }
I20260927 22:57:34.044797 7539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:34.044799 7540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:34.048161 7547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:34.053972 7547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:34.065681 7547 catalog_manager.cc:1347] Generated new cluster ID: 5d88c029e8db4eec88983f7756572e73
I20260927 22:57:34.066229 7547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:34.079188 7547 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:34.081080 7547 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:34.092590 7547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2795b79226bc469ca71b9f63a1f5db8f: Generated new TSK 0
I20260927 22:57:34.093818 7547 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:57:34.165035 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34919
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:34.509994 7559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:34.510291 7559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:34.510396 7559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:34.517699 7559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:34.517834 7559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:34.517951 7559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:57:34.528009 7559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:34.529434 7559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:34.530704 7559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:34.546638 7565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:34.546607 7567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:34.546839 7564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:34.547741 7559 server_base.cc:1034] running on GCE node
I20260927 22:57:34.548421 7559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:34.549510 7559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:34.550733 7559 hybrid_clock.cc:648] HybridClock initialized: now 1790549854550669 us; error 57 us; skew 500 ppm
I20260927 22:57:34.551102 7559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:34 dist-test-slave-5pd3 krb5kdc[7453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549854, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:34.554796 7559 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:57:34.556742 7559 webserver.cc:460] Webserver started at http://127.4.74.129:34825/ using document root <none> and password file <none>
I20260927 22:57:34.557562 7559 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:34.557678 7559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:34.558048 7559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:34.561177 7559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "c4b646ead6b1442bae95d8b374b0b4df"
format_stamp: "Formatted at 2026-09-27 22:57:34 on dist-test-slave-5pd3"
server_key: "7cde0834f0bf44628b336572ba12c874"
server_key_iv: "9fdeb650a8762d9ad5ae5ab941fdec59"
server_key_version: "encryptionkey@0"
I20260927 22:57:34.561980 7559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "c4b646ead6b1442bae95d8b374b0b4df"
format_stamp: "Formatted at 2026-09-27 22:57:34 on dist-test-slave-5pd3"
server_key: "7cde0834f0bf44628b336572ba12c874"
server_key_iv: "9fdeb650a8762d9ad5ae5ab941fdec59"
server_key_version: "encryptionkey@0"
I20260927 22:57:34.567770 7559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260927 22:57:34.572196 7574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:34.574095 7559 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 22:57:34.574363 7559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "c4b646ead6b1442bae95d8b374b0b4df"
format_stamp: "Formatted at 2026-09-27 22:57:34 on dist-test-slave-5pd3"
server_key: "7cde0834f0bf44628b336572ba12c874"
server_key_iv: "9fdeb650a8762d9ad5ae5ab941fdec59"
server_key_version: "encryptionkey@0"
I20260927 22:57:34.574569 7559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:34.590678 7559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:34.595184 7559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:34.595510 7559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:34.596621 7559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:34.598085 7559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:34.598196 7559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:34.598342 7559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:34.598412 7559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:34.638659 7559 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:46797
I20260927 22:57:34.638725 7686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:46797 every 8 connection(s)
I20260927 22:57:34.640278 7559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:57:34.641665 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 7559
I20260927 22:57:34.641952 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:57:34.642987 4394 external_mini_cluster.cc:1428] Setting key 56f4221eda956e48a1194f589038e25e
Sep 27 22:57:34 dist-test-slave-5pd3 krb5kdc[7453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549854, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
I20260927 22:57:34.670233 7689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:34.644502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:58121 (local address 127.4.74.190:34919)
0927 22:57:34.645122 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:34.645128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:34.646688 (+ 1560us) server_negotiation.cc:407] Connection header received
0927 22:57:34.649090 (+ 2402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:34.649097 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:34.649227 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:34.649371 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:34.652329 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:34.653547 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:34.655792 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:34.656618 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:34.656816 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:34.661894 (+ 5078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:34.661903 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:34.661910 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:34.661971 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:34.665287 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:34.666546 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:34.666555 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:34.666562 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:34.666742 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:34.667528 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:34.667532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:34.667535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:34.667962 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:34.668105 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:34.669571 (+ 1466us) server_negotiation.cc:299] Negotiation successful
0927 22:57:34.669787 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":299,"threads_started":1}
I20260927 22:57:34.672124 7690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:34.645298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34919 (local address 127.4.74.129:58121)
0927 22:57:34.646417 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:34.646518 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:34.648688 (+ 2170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:34.649721 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:34.649734 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:34.650455 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:34.651841 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:34.651884 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:34.653861 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:34.653867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:34.655475 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:34.655498 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:34.656962 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:34.656969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:34.657292 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:34.658247 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:34.658278 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:34.661449 (+ 3171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:34.665534 (+ 4085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:34.665546 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:34.665568 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:34.666330 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:34.666912 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:34.666918 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:34.666921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:34.667283 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:34.668129 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:34.668145 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:34.668913 (+ 768us) client_negotiation.cc:769] Sending connection context
0927 22:57:34.669346 (+ 433us) client_negotiation.cc:240] Negotiation successful
0927 22:57:34.670171 (+ 825us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":469,"threads_started":1}
I20260927 22:57:34.674458 7687 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34919
I20260927 22:57:34.674866 7687 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:34.675890 7687 heartbeater.cc:511] Master 127.4.74.190:34919 requested a full tablet report, sending...
I20260927 22:57:34.678803 7498 ts_manager.cc:194] Registered new tserver with Master: c4b646ead6b1442bae95d8b374b0b4df (127.4.74.129:46797)
I20260927 22:57:34.680663 7498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:58121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:57:34.699910 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34919
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:35.055364 7693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:35.055728 7693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:35.055891 7693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:35.063246 7693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:35.063426 7693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:35.063623 7693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:57:35.073555 7693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:35.074928 7693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:35.076305 7693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:35.091025 7698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:35.091181 7699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:35.091163 7701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:35.092043 7693 server_base.cc:1034] running on GCE node
I20260927 22:57:35.092694 7693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:35.093717 7693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:35.094981 7693 hybrid_clock.cc:648] HybridClock initialized: now 1790549855094915 us; error 62 us; skew 500 ppm
I20260927 22:57:35.095333 7693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:35 dist-test-slave-5pd3 krb5kdc[7453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549855, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:35.098802 7693 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:57:35.100541 7693 webserver.cc:460] Webserver started at http://127.4.74.130:35729/ using document root <none> and password file <none>
I20260927 22:57:35.101310 7693 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:35.101469 7693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:35.101804 7693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:35.104867 7693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "8cd4e99cc18f428ba0a472cf68310358"
format_stamp: "Formatted at 2026-09-27 22:57:35 on dist-test-slave-5pd3"
server_key: "6e016467bb903d88f9038900075e0ecc"
server_key_iv: "3feaeee94409981160f1bd03e1ff6110"
server_key_version: "encryptionkey@0"
I20260927 22:57:35.105672 7693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "8cd4e99cc18f428ba0a472cf68310358"
format_stamp: "Formatted at 2026-09-27 22:57:35 on dist-test-slave-5pd3"
server_key: "6e016467bb903d88f9038900075e0ecc"
server_key_iv: "3feaeee94409981160f1bd03e1ff6110"
server_key_version: "encryptionkey@0"
I20260927 22:57:35.111666 7693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260927 22:57:35.116230 7708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:35.118211 7693 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260927 22:57:35.118470 7693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "8cd4e99cc18f428ba0a472cf68310358"
format_stamp: "Formatted at 2026-09-27 22:57:35 on dist-test-slave-5pd3"
server_key: "6e016467bb903d88f9038900075e0ecc"
server_key_iv: "3feaeee94409981160f1bd03e1ff6110"
server_key_version: "encryptionkey@0"
I20260927 22:57:35.118711 7693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:35.139652 7693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:35.146060 7693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:35.146426 7693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:35.147691 7693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:35.149361 7693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:35.149446 7693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:35.149546 7693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:35.149598 7693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:35.188550 7693 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:34531
I20260927 22:57:35.188791 7820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:34531 every 8 connection(s)
I20260927 22:57:35.190199 7693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:57:35.194295 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 7693
I20260927 22:57:35.194556 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:57:35.195228 4394 external_mini_cluster.cc:1428] Setting key 442b4e4d91ba17a2d329a32a2d7424e6
Sep 27 22:57:35 dist-test-slave-5pd3 krb5kdc[7453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549855, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:35.216039 7823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:35.193306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:43951 (local address 127.4.74.190:34919)
0927 22:57:35.193819 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:35.193823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:35.195623 (+ 1800us) server_negotiation.cc:407] Connection header received
0927 22:57:35.197586 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:35.197597 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:35.197670 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:35.197793 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:35.199485 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.200279 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.201922 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.202353 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.202493 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:35.207196 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:35.207206 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:35.207220 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:35.207282 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:35.211318 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.212701 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.212710 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.212716 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.212901 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.213587 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.213595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.213600 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.214049 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:35.214222 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:35.215244 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0927 22:57:35.215522 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":303,"threads_started":1}
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 22:57:35.217295 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:35.193966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34919 (local address 127.4.74.130:43951)
0927 22:57:35.195159 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:35.195210 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:35.197184 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:35.198063 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:35.198072 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:35.198670 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:35.199297 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:35.199316 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.200458 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.200462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:35.201746 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:35.201760 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.202609 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.202616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:35.202858 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:35.203767 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:35.203808 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:35.206830 (+ 3022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:35.211672 (+ 4842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:35.211684 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:35.211705 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:35.212512 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:35.213129 (+ 617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:35.213133 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:35.213136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:35.213422 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:35.214242 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:35.214251 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:35.214757 (+ 506us) client_negotiation.cc:769] Sending connection context
0927 22:57:35.215086 (+ 329us) client_negotiation.cc:240] Negotiation successful
0927 22:57:35.215724 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"thread_start_us":458,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
I20260927 22:57:35.219612 7821 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34919
I20260927 22:57:35.220098 7821 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:35.221313 7821 heartbeater.cc:511] Master 127.4.74.190:34919 requested a full tablet report, sending...
I20260927 22:57:35.223735 7498 ts_manager.cc:194] Registered new tserver with Master: 8cd4e99cc18f428ba0a472cf68310358 (127.4.74.130:34531)
I20260927 22:57:35.224743 7498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:43951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:57:35.248734 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34919
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:44515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:35.581584 7827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:35.581903 7827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:35.582006 7827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:35.589625 7827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:35.589756 7827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:35.589869 7827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:57:35.599854 7827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:44515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:35.601186 7827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:35.602468 7827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:35.617712 7832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:35.617967 7835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:35.617980 7833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:35.619323 7827 server_base.cc:1034] running on GCE node
I20260927 22:57:35.620163 7827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:35.621268 7827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:35.622553 7827 hybrid_clock.cc:648] HybridClock initialized: now 1790549855622449 us; error 48 us; skew 500 ppm
I20260927 22:57:35.623034 7827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:35 dist-test-slave-5pd3 krb5kdc[7453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549855, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:35.626715 7827 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:57:35.628530 7827 webserver.cc:460] Webserver started at http://127.4.74.131:35399/ using document root <none> and password file <none>
I20260927 22:57:35.629288 7827 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:35.629438 7827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:35.629746 7827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:35.632841 7827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "b298cfee16ec4374938deed5f5fe13db"
format_stamp: "Formatted at 2026-09-27 22:57:35 on dist-test-slave-5pd3"
server_key: "04546f457956bed1392d6e0a7d5457dc"
server_key_iv: "16bf49a88c870b74090824a42c4008cf"
server_key_version: "encryptionkey@0"
I20260927 22:57:35.633699 7827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "b298cfee16ec4374938deed5f5fe13db"
format_stamp: "Formatted at 2026-09-27 22:57:35 on dist-test-slave-5pd3"
server_key: "04546f457956bed1392d6e0a7d5457dc"
server_key_iv: "16bf49a88c870b74090824a42c4008cf"
server_key_version: "encryptionkey@0"
I20260927 22:57:35.640085 7827 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 22:57:35.644757 7842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:35.646337 7827 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260927 22:57:35.646551 7827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "b298cfee16ec4374938deed5f5fe13db"
format_stamp: "Formatted at 2026-09-27 22:57:35 on dist-test-slave-5pd3"
server_key: "04546f457956bed1392d6e0a7d5457dc"
server_key_iv: "16bf49a88c870b74090824a42c4008cf"
server_key_version: "encryptionkey@0"
I20260927 22:57:35.646775 7827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:35.670921 7827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:35.675642 7827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:35.675976 7827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:35.677083 7827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:35.678700 7827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:35.678810 7827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:35.678961 7827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:35.679035 7827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:35.685485 7687 heartbeater.cc:503] Master 127.4.74.190:34919 was elected leader, sending a full tablet report...
I20260927 22:57:35.720971 7827 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:40733
I20260927 22:57:35.721072 7954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:40733 every 8 connection(s)
I20260927 22:57:35.722319 7827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:57:35.732561 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 7827
I20260927 22:57:35.732803 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:57:35.733325 4394 external_mini_cluster.cc:1428] Setting key 2e7e456f537c94fb13074420577e7df6
Sep 27 22:57:35 dist-test-slave-5pd3 krb5kdc[7453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549855, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:35.747005 7957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:35.725654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:34285 (local address 127.4.74.190:34919)
0927 22:57:35.726163 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:35.726170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:35.727522 (+ 1352us) server_negotiation.cc:407] Connection header received
0927 22:57:35.729473 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:35.729478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:35.729566 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:35.729716 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:35.731377 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.732604 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.734628 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.735223 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.735404 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:35.739490 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:35.739496 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:35.739500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:35.739623 (+ 123us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:35.742834 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.744101 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.744107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.744109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.744268 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.744836 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.744840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.744842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.745166 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:35.745304 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:35.746344 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0927 22:57:35.746555 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":226,"threads_started":1}
I20260927 22:57:35.748354 7958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:35.726352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34919 (local address 127.4.74.131:34285)
0927 22:57:35.727297 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:35.727377 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:35.729159 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:35.729947 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:35.729956 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:35.730529 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:35.731132 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:35.731150 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.732873 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.732882 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:35.734468 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:35.734481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.735502 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.735510 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:35.735848 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:35.736671 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:35.736694 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:35.739210 (+ 2516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:35.743124 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:35.743136 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:35.743156 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:35.743926 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:35.744424 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:35.744428 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:35.744431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:35.744669 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:35.745331 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:35.745339 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:35.745833 (+ 494us) client_negotiation.cc:769] Sending connection context
0927 22:57:35.746158 (+ 325us) client_negotiation.cc:240] Negotiation successful
0927 22:57:35.746714 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":356,"threads_started":1}
I20260927 22:57:35.750313 7955 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34919
I20260927 22:57:35.750751 7955 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:35.751686 7955 heartbeater.cc:511] Master 127.4.74.190:34919 requested a full tablet report, sending...
I20260927 22:57:35.753350 7498 ts_manager.cc:194] Registered new tserver with Master: b298cfee16ec4374938deed5f5fe13db (127.4.74.131:40733)
I20260927 22:57:35.754273 7498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:34285
I20260927 22:57:35.763725 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 22:57:35 dist-test-slave-5pd3 krb5kdc[7453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549855, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 27 22:57:35 dist-test-slave-5pd3 krb5kdc[7453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549855, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:35.822365 7957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:35.805430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.4.74.190:34919)
0927 22:57:35.805884 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:35.805895 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:35.806483 (+ 588us) server_negotiation.cc:407] Connection header received
0927 22:57:35.806760 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:35.806764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:35.806855 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:35.806967 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:35.808586 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.809402 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.810699 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.811130 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.811265 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:35.816022 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:35.816030 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:35.816035 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:35.816085 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:35.818971 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.820094 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.820098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.820101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.820246 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.820693 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.820697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.820700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.821118 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:35.821344 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:35.821824 (+ 480us) server_negotiation.cc:299] Negotiation successful
0927 22:57:35.822105 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
W20260927 22:57:35.823143 7498 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42924
I20260927 22:57:35.826086 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7559
Sep 27 22:57:35 dist-test-slave-5pd3 krb5kdc[7453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549855, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:35.847203 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:35.827467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.4.74.129:46797)
0927 22:57:35.828191 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:35.828199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:35.828228 (+ 29us) server_negotiation.cc:407] Connection header received
0927 22:57:35.828300 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:35.828306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:35.828744 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:35.828933 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:35.830450 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.831524 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.833116 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.833887 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.834023 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:35.839231 (+ 5208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:35.839254 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:35.839276 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:35.839355 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:35.843877 (+ 4522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.845099 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.845108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.845111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.845290 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.845653 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.845656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.845658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.846067 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:35.846368 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:35.846596 (+ 228us) server_negotiation.cc:299] Negotiation successful
0927 22:57:35.846859 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":368,"threads_started":1}
W20260927 22:57:35.849125 7682 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40028
W20260927 22:57:35.850754 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7559: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 22:57:35.850965 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 7559
I20260927 22:57:35.867419 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7693
Sep 27 22:57:35 dist-test-slave-5pd3 krb5kdc[7453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549855, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:57:35.887051 7968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:35.868787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32932 (local address 127.4.74.130:34531)
0927 22:57:35.869379 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:35.869387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:35.869434 (+ 47us) server_negotiation.cc:407] Connection header received
0927 22:57:35.869510 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:35.869517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:35.869748 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:35.869886 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:35.871520 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.872678 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.874247 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.874854 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.874985 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:35.880088 (+ 5103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:35.880105 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:35.880123 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:35.880168 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:35.883434 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.884639 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.884647 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.884649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.884814 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.885284 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.885288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.885290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.885663 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:35.885911 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:35.886386 (+ 475us) server_negotiation.cc:299] Negotiation successful
0927 22:57:35.886635 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":289,"threads_started":1}
W20260927 22:57:35.888854 7816 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32932
W20260927 22:57:35.890268 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7693: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 22:57:35.890543 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 7693
I20260927 22:57:35.907936 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7827
Sep 27 22:57:35 dist-test-slave-5pd3 krb5kdc[7453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549855, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:57:35.928052 7969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:35.909136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53594 (local address 127.4.74.131:40733)
0927 22:57:35.909926 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:35.909934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:35.909965 (+ 31us) server_negotiation.cc:407] Connection header received
0927 22:57:35.910034 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:35.910041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:35.910275 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:35.910407 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:35.912033 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.913101 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.914516 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:35.915145 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:35.915278 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:35.921043 (+ 5765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:35.921058 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:35.921074 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:35.921123 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:35.924185 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.925021 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.925033 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.925035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.925171 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:35.925947 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:35.925951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:35.925953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:35.926325 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:35.926593 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:35.927049 (+ 456us) server_negotiation.cc:299] Negotiation successful
0927 22:57:35.927495 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":412,"threads_started":1}
W20260927 22:57:35.929836 7950 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53594
W20260927 22:57:35.931315 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7827: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 22:57:35.931499 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 7827
I20260927 22:57:35.948009 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7466
I20260927 22:57:35.949487 7528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:36.060452 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 7466
2026-09-27T22:57:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3964 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:36 dist-test-slave-5pd3 krb5kdc[7973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:36 dist-test-slave-5pd3 krb5kdc[7973](info): set up 2 sockets
Sep 27 22:57:36 dist-test-slave-5pd3 krb5kdc[7973](info): commencing operation
krb5kdc: starting...
W20260927 22:57:37.184231 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:37 dist-test-slave-5pd3 krb5kdc[7973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:57:37.413096 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:39231
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:39231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:37.785259 7986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:37.785635 7986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:37.785789 7986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:37.793462 7986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:37.793630 7986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:37.793738 7986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:37.793833 7986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:37.793927 7986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:37.804054 7986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:39231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:39231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:37.805593 7986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:37.806991 7986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:37.820542 7994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:37.820894 7992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:37.821170 7991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:37.822171 7986 server_base.cc:1034] running on GCE node
I20260927 22:57:37.823128 7986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:37.824815 7986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:37.826082 7986 hybrid_clock.cc:648] HybridClock initialized: now 1790549857826006 us; error 68 us; skew 500 ppm
I20260927 22:57:37.826472 7986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:37 dist-test-slave-5pd3 krb5kdc[7973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549857, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:37.843428 7986 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:57:37.845674 7986 webserver.cc:460] Webserver started at http://127.4.74.190:37787/ using document root <none> and password file <none>
I20260927 22:57:37.846495 7986 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:37.846640 7986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:37.846989 7986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:37.850281 7986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e"
format_stamp: "Formatted at 2026-09-27 22:57:37 on dist-test-slave-5pd3"
server_key: "49dd56b5ded2cc082a45fd638238e681"
server_key_iv: "e94f412048d6eb0a032544e24195e38d"
server_key_version: "encryptionkey@0"
I20260927 22:57:37.851150 7986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e"
format_stamp: "Formatted at 2026-09-27 22:57:37 on dist-test-slave-5pd3"
server_key: "49dd56b5ded2cc082a45fd638238e681"
server_key_iv: "e94f412048d6eb0a032544e24195e38d"
server_key_version: "encryptionkey@0"
I20260927 22:57:37.856902 7986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260927 22:57:37.860816 8001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:37.862665 7986 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:57:37.862920 7986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e"
format_stamp: "Formatted at 2026-09-27 22:57:37 on dist-test-slave-5pd3"
server_key: "49dd56b5ded2cc082a45fd638238e681"
server_key_iv: "e94f412048d6eb0a032544e24195e38d"
server_key_version: "encryptionkey@0"
I20260927 22:57:37.863142 7986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:37.884583 7986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:37.889065 7986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:37.889402 7986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:37.912134 7986 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:39231
I20260927 22:57:37.912206 8052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:39231 every 8 connection(s)
I20260927 22:57:37.913671 7986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:37.916970 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 7986
I20260927 22:57:37.917173 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:37.917878 4394 external_mini_cluster.cc:1428] Setting key 63f77c9ff4f8e622006fd749a812ccab
I20260927 22:57:37.918980 8053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 22:57:37 dist-test-slave-5pd3 krb5kdc[7973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:37.936805 8053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e: Bootstrap starting.
I20260927 22:57:37.942225 8053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:37.944177 8053 log.cc:824] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:37.947013 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:37.920389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52768 (local address 127.4.74.190:39231)
0927 22:57:37.921356 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:37.921373 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:37.921437 (+ 64us) server_negotiation.cc:407] Connection header received
0927 22:57:37.923455 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:37.923493 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:37.924239 (+ 746us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:37.924825 (+ 586us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:37.926362 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:37.928795 (+ 2433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:37.930330 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:37.931593 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:37.931879 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:37.936269 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:37.936283 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:37.936312 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:37.936380 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:37.940913 (+ 4533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:37.941853 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:37.941864 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:37.941877 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:37.942171 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:37.942635 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:37.942642 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:37.942647 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:37.943860 (+ 1213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:37.944276 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:37.944694 (+ 418us) server_negotiation.cc:299] Negotiation successful
0927 22:57:37.945411 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":381,"threads_started":1}
I20260927 22:57:37.948563 8053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e: No bootstrap required, opened a new log
I20260927 22:57:37.958786 8053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39231 } }
I20260927 22:57:37.959309 8053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:37.959430 8053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc0bbe70c464b5d9b13d2f4ec66161e, State: Initialized, Role: FOLLOWER
I20260927 22:57:37.960309 8053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39231 } }
I20260927 22:57:37.960637 8053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:37.960876 8053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:37.961098 8053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:37.963779 8053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39231 } }
I20260927 22:57:37.964493 8053 leader_election.cc:302] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fc0bbe70c464b5d9b13d2f4ec66161e; no voters:
I20260927 22:57:37.965315 8053 leader_election.cc:288] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:37.965513 8058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:37.966526 8058 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [term 1 LEADER]: Becoming Leader. State: Replica: 3fc0bbe70c464b5d9b13d2f4ec66161e, State: Running, Role: LEADER
I20260927 22:57:37.967247 8058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39231 } }
I20260927 22:57:37.968132 8053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:37.970963 8059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39231 } } }
I20260927 22:57:37.971694 8059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:37.971120 8060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fc0bbe70c464b5d9b13d2f4ec66161e. Latest consensus state: current_term: 1 leader_uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc0bbe70c464b5d9b13d2f4ec66161e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39231 } } }
I20260927 22:57:37.972919 8060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:37.974545 8066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:37.980829 8066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:37.992812 8066 catalog_manager.cc:1347] Generated new cluster ID: 7d04f422ff544fdfbdc12d0f1bb878d6
I20260927 22:57:37.993009 8066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:38.004149 8066 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:38.005735 8066 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:38.017375 8066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fc0bbe70c464b5d9b13d2f4ec66161e: Generated new TSK 0
I20260927 22:57:38.019060 8066 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:57:38.085764 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39231
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:38.453578 8079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:38.453872 8079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:38.454020 8079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:38.461443 8079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:38.461567 8079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:38.461737 8079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:57:38.472036 8079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:38.473436 8079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:38.474785 8079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:38.489089 8084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:38.489306 8087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:38.489315 8085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:38.490962 8079 server_base.cc:1034] running on GCE node
I20260927 22:57:38.491858 8079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:38.493117 8079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:38.494352 8079 hybrid_clock.cc:648] HybridClock initialized: now 1790549858494268 us; error 72 us; skew 500 ppm
I20260927 22:57:38.494777 8079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:38 dist-test-slave-5pd3 krb5kdc[7973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549858, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:38.498986 8079 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:57:38.501246 8079 webserver.cc:460] Webserver started at http://127.4.74.129:46877/ using document root <none> and password file <none>
I20260927 22:57:38.502177 8079 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:38.502332 8079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:38.502722 8079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:38.506537 8079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "3dd08baca4464dc399b9d59c09aa0f4e"
format_stamp: "Formatted at 2026-09-27 22:57:38 on dist-test-slave-5pd3"
server_key: "f9c04ed92cc95c148877046615bc69a1"
server_key_iv: "6f8365fafb47322f7d40e70671479c69"
server_key_version: "encryptionkey@0"
I20260927 22:57:38.507666 8079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "3dd08baca4464dc399b9d59c09aa0f4e"
format_stamp: "Formatted at 2026-09-27 22:57:38 on dist-test-slave-5pd3"
server_key: "f9c04ed92cc95c148877046615bc69a1"
server_key_iv: "6f8365fafb47322f7d40e70671479c69"
server_key_version: "encryptionkey@0"
I20260927 22:57:38.515161 8079 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260927 22:57:38.520218 8094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:38.522328 8079 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 22:57:38.522576 8079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "3dd08baca4464dc399b9d59c09aa0f4e"
format_stamp: "Formatted at 2026-09-27 22:57:38 on dist-test-slave-5pd3"
server_key: "f9c04ed92cc95c148877046615bc69a1"
server_key_iv: "6f8365fafb47322f7d40e70671479c69"
server_key_version: "encryptionkey@0"
I20260927 22:57:38.522851 8079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:38.547101 8079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:38.552776 8079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:38.553144 8079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:38.554355 8079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:38.556071 8079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:38.556188 8079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:38.556342 8079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:38.556416 8079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:38.596244 8079 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:39917
I20260927 22:57:38.596315 8206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:39917 every 8 connection(s)
I20260927 22:57:38.597946 8079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:57:38.602663 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 8079
I20260927 22:57:38.602931 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:57:38.603775 4394 external_mini_cluster.cc:1428] Setting key d3ea64f306e3763ea25d2e4c3f96438b
Sep 27 22:57:38 dist-test-slave-5pd3 krb5kdc[7973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549858, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:38.623769 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:38.601305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:44891 (local address 127.4.74.190:39231)
0927 22:57:38.601885 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:38.601890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:38.603227 (+ 1337us) server_negotiation.cc:407] Connection header received
0927 22:57:38.605680 (+ 2453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:38.605687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:38.605835 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:38.605977 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:38.607936 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:38.608726 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:38.610469 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:38.610921 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:38.611064 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:38.616215 (+ 5151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:38.616221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:38.616225 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:38.616267 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:38.619272 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:38.620503 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:38.620513 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:38.620542 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:38.620763 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:38.621366 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:38.621374 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:38.621380 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:38.621852 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:38.622067 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:38.622924 (+ 857us) server_negotiation.cc:299] Negotiation successful
0927 22:57:38.623223 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":300,"threads_started":1}
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 22:57:38.624576 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:38.601994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39231 (local address 127.4.74.129:44891)
0927 22:57:38.602988 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:38.603053 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:38.605336 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:38.606279 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:38.606290 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:38.606940 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:38.607730 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:38.607761 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:38.608906 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:38.608917 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:38.610298 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:38.610323 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:38.611190 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:38.611200 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:38.611474 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:38.612358 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:38.612385 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:38.615888 (+ 3503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:38.619504 (+ 3616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:38.619518 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:38.619582 (+ 64us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:38.620278 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:38.620957 (+ 679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:38.620962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:38.620964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:38.621209 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:38.622007 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:38.622014 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:38.622442 (+ 428us) client_negotiation.cc:769] Sending connection context
0927 22:57:38.622711 (+ 269us) client_negotiation.cc:240] Negotiation successful
0927 22:57:38.623175 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":422,"threads_started":1}
I20260927 22:57:38.626367 8207 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39231
I20260927 22:57:38.626722 8207 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
I20260927 22:57:38.627640 8207 heartbeater.cc:511] Master 127.4.74.190:39231 requested a full tablet report, sending...
I20260927 22:57:38.630128 8018 ts_manager.cc:194] Registered new tserver with Master: 3dd08baca4464dc399b9d59c09aa0f4e (127.4.74.129:39917)
I20260927 22:57:38.631719 8018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:44891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:57:38.655097 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39231
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:38.994365 8213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:38.994697 8213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:38.994846 8213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:39.003062 8213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:39.003237 8213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:39.003408 8213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:57:39.014294 8213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:39.015796 8213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:39.017225 8213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:39.036814 8219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:39.037176 8221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:39.037715 8213 server_base.cc:1034] running on GCE node
W20260927 22:57:39.038028 8218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:39.039091 8213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:39.040619 8213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:39.041872 8213 hybrid_clock.cc:648] HybridClock initialized: now 1790549859041800 us; error 74 us; skew 500 ppm
I20260927 22:57:39.042317 8213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:39 dist-test-slave-5pd3 krb5kdc[7973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549859, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:39.048108 8213 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:57:39.050417 8213 webserver.cc:460] Webserver started at http://127.4.74.130:40437/ using document root <none> and password file <none>
I20260927 22:57:39.051489 8213 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:39.051699 8213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:39.052127 8213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:39.056533 8213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "630a489ddc864f63a238c21913bea6f5"
format_stamp: "Formatted at 2026-09-27 22:57:39 on dist-test-slave-5pd3"
server_key: "1bea138e1c7065d9bccb9c669393c227"
server_key_iv: "acbbc942d2577588e259369e9a3790b9"
server_key_version: "encryptionkey@0"
I20260927 22:57:39.057597 8213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "630a489ddc864f63a238c21913bea6f5"
format_stamp: "Formatted at 2026-09-27 22:57:39 on dist-test-slave-5pd3"
server_key: "1bea138e1c7065d9bccb9c669393c227"
server_key_iv: "acbbc942d2577588e259369e9a3790b9"
server_key_version: "encryptionkey@0"
I20260927 22:57:39.066435 8213 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260927 22:57:39.072839 8228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:39.075196 8213 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260927 22:57:39.075435 8213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "630a489ddc864f63a238c21913bea6f5"
format_stamp: "Formatted at 2026-09-27 22:57:39 on dist-test-slave-5pd3"
server_key: "1bea138e1c7065d9bccb9c669393c227"
server_key_iv: "acbbc942d2577588e259369e9a3790b9"
server_key_version: "encryptionkey@0"
I20260927 22:57:39.075769 8213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:39.112571 8213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:39.117354 8213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:39.117671 8213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:39.118785 8213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:39.120404 8213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:39.120512 8213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:39.120669 8213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:39.120744 8213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:39.159356 8213 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:40165
I20260927 22:57:39.159457 8340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:40165 every 8 connection(s)
I20260927 22:57:39.160822 8213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:57:39.170794 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 8213
I20260927 22:57:39.171172 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:57:39.171819 4394 external_mini_cluster.cc:1428] Setting key 31c039a4365a4ff396e1b64cb9b9e80d
Sep 27 22:57:39 dist-test-slave-5pd3 krb5kdc[7973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549859, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:39.187404 8343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:39.164129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:42331 (local address 127.4.74.190:39231)
0927 22:57:39.164653 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:39.164659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:39.165948 (+ 1289us) server_negotiation.cc:407] Connection header received
0927 22:57:39.168447 (+ 2499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:39.168454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:39.168557 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:39.168681 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:39.170375 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.171175 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.173676 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.174298 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.174412 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:39.180062 (+ 5650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:39.180073 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:39.180081 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:39.180140 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:39.183732 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:39.184827 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:39.184832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:39.184834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:39.185012 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:39.185521 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:39.185524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:39.185526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:39.185823 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:39.185962 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:39.186897 (+ 935us) server_negotiation.cc:299] Negotiation successful
0927 22:57:39.187093 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":258,"threads_started":1}
I20260927 22:57:39.189016 8344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:39.164815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39231 (local address 127.4.74.130:42331)
0927 22:57:39.165733 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:39.165792 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:39.168067 (+ 2275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:39.168962 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:39.168972 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:39.169584 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:39.170188 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:39.170205 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.171482 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.171503 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:39.173474 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:39.173499 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.174594 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.174603 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:39.174983 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:39.175938 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:39.175988 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:39.179717 (+ 3729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:39.183912 (+ 4195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:39.183932 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:39.183955 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:39.184666 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:39.185133 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:39.185140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:39.185147 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:39.185398 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:39.185983 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:39.185994 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:39.186481 (+ 487us) client_negotiation.cc:769] Sending connection context
0927 22:57:39.186786 (+ 305us) client_negotiation.cc:240] Negotiation successful
0927 22:57:39.187388 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":364,"threads_started":1}
I20260927 22:57:39.191219 8341 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39231
I20260927 22:57:39.191708 8341 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:39.192667 8341 heartbeater.cc:511] Master 127.4.74.190:39231 requested a full tablet report, sending...
I20260927 22:57:39.194648 8018 ts_manager.cc:194] Registered new tserver with Master: 630a489ddc864f63a238c21913bea6f5 (127.4.74.130:40165)
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 22:57:39.195753 8018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:42331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:57:39.224447 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39231
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:57:39.635144 8207 heartbeater.cc:503] Master 127.4.74.190:39231 was elected leader, sending a full tablet report...
W20260927 22:57:39.638685 8347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:39.639097 8347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:39.639264 8347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:39.648094 8347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:39.648321 8347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:39.648542 8347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:57:39.661494 8347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:39.663391 8347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:39.665210 8347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:39.682117 8353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:39.682163 8355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:39.682318 8352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:39.683930 8347 server_base.cc:1034] running on GCE node
I20260927 22:57:39.684871 8347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:39.686195 8347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:39.687460 8347 hybrid_clock.cc:648] HybridClock initialized: now 1790549859687367 us; error 74 us; skew 500 ppm
I20260927 22:57:39.687925 8347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:39 dist-test-slave-5pd3 krb5kdc[7973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549859, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:39.691969 8347 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:57:39.693908 8347 webserver.cc:460] Webserver started at http://127.4.74.131:45103/ using document root <none> and password file <none>
I20260927 22:57:39.694756 8347 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:39.694871 8347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:39.695289 8347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:39.698660 8347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "c9772683cbfb43ffb2ebccd32c342d2e"
format_stamp: "Formatted at 2026-09-27 22:57:39 on dist-test-slave-5pd3"
server_key: "88e292ed568bf1e545d279eef7d17c59"
server_key_iv: "1f23c5fdea1d9416c7779e1f6e3021d8"
server_key_version: "encryptionkey@0"
I20260927 22:57:39.699716 8347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "c9772683cbfb43ffb2ebccd32c342d2e"
format_stamp: "Formatted at 2026-09-27 22:57:39 on dist-test-slave-5pd3"
server_key: "88e292ed568bf1e545d279eef7d17c59"
server_key_iv: "1f23c5fdea1d9416c7779e1f6e3021d8"
server_key_version: "encryptionkey@0"
I20260927 22:57:39.705907 8347 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 22:57:39.710619 8362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:39.712679 8347 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 22:57:39.712929 8347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "c9772683cbfb43ffb2ebccd32c342d2e"
format_stamp: "Formatted at 2026-09-27 22:57:39 on dist-test-slave-5pd3"
server_key: "88e292ed568bf1e545d279eef7d17c59"
server_key_iv: "1f23c5fdea1d9416c7779e1f6e3021d8"
server_key_version: "encryptionkey@0"
I20260927 22:57:39.713184 8347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:39.742323 8347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:39.748272 8347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:39.748652 8347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:39.749966 8347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:39.751935 8347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:39.752053 8347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:39.752259 8347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:39.752333 8347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:39.796876 8347 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:42237
I20260927 22:57:39.796926 8474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:42237 every 8 connection(s)
I20260927 22:57:39.798529 8347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:57:39.800388 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 8347
I20260927 22:57:39.800622 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:57:39.801213 4394 external_mini_cluster.cc:1428] Setting key a2c8b8c77ca1dbcf6ff853c4ddfb5673
Sep 27 22:57:39 dist-test-slave-5pd3 krb5kdc[7973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549859, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:39.824362 8477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:39.801946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:50629 (local address 127.4.74.190:39231)
0927 22:57:39.802446 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:39.802450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:39.804797 (+ 2347us) server_negotiation.cc:407] Connection header received
0927 22:57:39.806317 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:39.806321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:39.806405 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:39.806630 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:39.808641 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.809476 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.811356 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.811870 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.811992 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:39.816380 (+ 4388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:39.816386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:39.816391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:39.816439 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:39.820335 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:39.821397 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:39.821402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:39.821405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:39.821554 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:39.822191 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:39.822195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:39.822198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:39.822534 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:39.822676 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:39.823785 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0927 22:57:39.824042 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":276,"threads_started":1}
I20260927 22:57:39.825621 8478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:39.802714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39231 (local address 127.4.74.131:50629)
0927 22:57:39.803783 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:39.803854 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:39.805983 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:39.806862 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:39.806872 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:39.807670 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:39.808414 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:39.808436 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.809672 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.809678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:39.811135 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:39.811157 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.812142 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.812148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:39.812438 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:39.813288 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:39.813316 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:39.816128 (+ 2812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:39.820563 (+ 4435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:39.820573 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:39.820593 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:39.821228 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:39.821729 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:39.821736 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:39.821739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:39.822005 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:39.822715 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:39.822723 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:39.823217 (+ 494us) client_negotiation.cc:769] Sending connection context
0927 22:57:39.823612 (+ 395us) client_negotiation.cc:240] Negotiation successful
0927 22:57:39.824179 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":415,"threads_started":1}
I20260927 22:57:39.827633 8475 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39231
I20260927 22:57:39.828092 8475 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:39.829015 8475 heartbeater.cc:511] Master 127.4.74.190:39231 requested a full tablet report, sending...
I20260927 22:57:39.831079 8018 ts_manager.cc:194] Registered new tserver with Master: c9772683cbfb43ffb2ebccd32c342d2e (127.4.74.131:42237)
I20260927 22:57:39.832556 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 22:57:39.832548 8018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:50629
Sep 27 22:57:39 dist-test-slave-5pd3 krb5kdc[7973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 22:57:39 dist-test-slave-5pd3 krb5kdc[7973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:39.894428 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:39.875814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.4.74.129:39917)
0927 22:57:39.876596 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:39.876608 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:39.876646 (+ 38us) server_negotiation.cc:407] Connection header received
0927 22:57:39.876743 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:39.876751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:39.877037 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:39.877255 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:39.878848 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.880049 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.881416 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.882162 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.882322 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:39.887228 (+ 4906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:39.887245 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:39.887265 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:39.887318 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:39.890687 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:39.891712 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:39.891720 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:39.891724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:39.891909 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:39.892353 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:39.892357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:39.892359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:39.892810 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:39.893089 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:39.893478 (+ 389us) server_negotiation.cc:299] Negotiation successful
0927 22:57:39.893745 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":362,"threads_started":1}
Sep 27 22:57:39 dist-test-slave-5pd3 krb5kdc[7973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:39.918926 8477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:39.900208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.4.74.190:39231)
0927 22:57:39.900515 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:39.900523 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:39.900774 (+ 251us) server_negotiation.cc:407] Connection header received
0927 22:57:39.901019 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:39.901023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:39.901094 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:39.901204 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:39.902680 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.903624 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.904987 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.905424 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.905570 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:39.910661 (+ 5091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:39.910668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:39.910673 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:39.910720 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:39.915135 (+ 4415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:39.916331 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:39.916341 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:39.916347 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:39.916542 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:39.916990 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:39.916995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:39.916999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:39.917459 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:39.917839 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:39.918200 (+ 361us) server_negotiation.cc:299] Negotiation successful
0927 22:57:39.918525 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260927 22:57:39.919920 8018 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52770
I20260927 22:57:39.921406 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 8079
I20260927 22:57:39.937675 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:39.922629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39466 (local address 127.4.74.129:39917)
0927 22:57:39.922868 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:39.922879 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:39.923188 (+ 309us) server_negotiation.cc:407] Connection header received
0927 22:57:39.923415 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:39.923422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:39.923583 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:39.923700 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:39.925067 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.925837 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.927393 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:39.928127 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:39.928309 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:39.931321 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:39.931327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:39.931331 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:39.931380 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:39.934563 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:39.935374 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:39.935380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:39.935384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:39.935595 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:39.936015 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:39.936022 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:39.936025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:39.936342 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:39.936617 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:39.936975 (+ 358us) server_negotiation.cc:299] Negotiation successful
0927 22:57:39.937295 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 22:57:39.938488 8202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:40.046384 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 8079
I20260927 22:57:40.070545 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 8213
Sep 27 22:57:40 dist-test-slave-5pd3 krb5kdc[7973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:57:40.094439 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:40.072048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55458 (local address 127.4.74.130:40165)
0927 22:57:40.072906 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:40.072916 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:40.072950 (+ 34us) server_negotiation.cc:407] Connection header received
0927 22:57:40.073041 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:40.073049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:40.073343 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:40.073663 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:40.076133 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:40.077473 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:40.079898 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:40.080676 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:40.080827 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:40.086557 (+ 5730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:40.086579 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:40.086603 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:40.086661 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:40.090301 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:40.091217 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:40.091226 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:40.091229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:40.091421 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:40.092205 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:40.092210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:40.092212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:40.092698 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:40.093105 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:40.093643 (+ 538us) server_negotiation.cc:299] Negotiation successful
0927 22:57:40.094000 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":480,"threads_started":1}
I20260927 22:57:40.096485 8336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:40.199239 8341 heartbeater.cc:503] Master 127.4.74.190:39231 was elected leader, sending a full tablet report...
I20260927 22:57:40.200064 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 8213
W20260927 22:57:40.213356 8002 connection.cc:535] server connection from 127.4.74.130:42331 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 22:57:40.213759 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 8347
Sep 27 22:57:40 dist-test-slave-5pd3 krb5kdc[7973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:57:40.234792 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:40.214949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48322 (local address 127.4.74.131:42237)
0927 22:57:40.215725 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:40.215734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:40.215773 (+ 39us) server_negotiation.cc:407] Connection header received
0927 22:57:40.215868 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:40.215877 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:40.216194 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:40.216365 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:40.218073 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:40.219377 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:40.221189 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:40.222126 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:40.222312 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:40.227411 (+ 5099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:40.227433 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:40.227453 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:40.227511 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:40.231236 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:40.232237 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:40.232245 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:40.232248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:40.232395 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:40.232810 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:40.232815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:40.232818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:40.233294 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:40.233572 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:40.233956 (+ 384us) server_negotiation.cc:299] Negotiation successful
0927 22:57:40.234345 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":297,"threads_started":1}
I20260927 22:57:40.236529 8470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:40.332067 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 8347
I20260927 22:57:40.348364 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 7986
I20260927 22:57:40.349941 8048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:40.458451 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 7986
2026-09-27T22:57:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4399 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:40 dist-test-slave-5pd3 krb5kdc[8495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:40 dist-test-slave-5pd3 krb5kdc[8495](info): set up 2 sockets
Sep 27 22:57:40 dist-test-slave-5pd3 krb5kdc[8495](info): commencing operation
krb5kdc: starting...
W20260927 22:57:41.613215 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:41 dist-test-slave-5pd3 krb5kdc[8495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:57:41.856557 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:41043
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:41043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:42.252375 8508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:42.252797 8508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:42.252969 8508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:42.260764 8508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:42.260962 8508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:42.261044 8508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:42.261125 8508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:42.261214 8508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:42.271418 8508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:41043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:41043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:42.272881 8508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:42.274276 8508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:42.288447 8513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:42.288709 8516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:42.288954 8514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:42.290118 8508 server_base.cc:1034] running on GCE node
I20260927 22:57:42.291141 8508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:42.292968 8508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:42.294288 8508 hybrid_clock.cc:648] HybridClock initialized: now 1790549862294231 us; error 59 us; skew 500 ppm
I20260927 22:57:42.294716 8508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:42 dist-test-slave-5pd3 krb5kdc[8495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549862, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:42.298614 8508 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:57:42.300068 8508 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:42.300166 8508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:42.300490 8508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:42.303750 8508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "4a3816d958694d779a0e9a6e044437fe"
format_stamp: "Formatted at 2026-09-27 22:57:42 on dist-test-slave-5pd3"
server_key: "5fc1d59fef3520ca0359d8895266a0c5"
server_key_iv: "fbccc9df3937ded0bfbd5f9726ee439c"
server_key_version: "encryptionkey@0"
I20260927 22:57:42.304648 8508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "4a3816d958694d779a0e9a6e044437fe"
format_stamp: "Formatted at 2026-09-27 22:57:42 on dist-test-slave-5pd3"
server_key: "5fc1d59fef3520ca0359d8895266a0c5"
server_key_iv: "fbccc9df3937ded0bfbd5f9726ee439c"
server_key_version: "encryptionkey@0"
I20260927 22:57:42.310528 8508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 22:57:42.314728 8522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:42.316708 8508 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:57:42.316957 8508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "4a3816d958694d779a0e9a6e044437fe"
format_stamp: "Formatted at 2026-09-27 22:57:42 on dist-test-slave-5pd3"
server_key: "5fc1d59fef3520ca0359d8895266a0c5"
server_key_iv: "fbccc9df3937ded0bfbd5f9726ee439c"
server_key_version: "encryptionkey@0"
I20260927 22:57:42.317219 8508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:42.349537 8508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:42.354445 8508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:42.354799 8508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:42.377812 8508 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:41043
I20260927 22:57:42.377738 8573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:41043 every 8 connection(s)
I20260927 22:57:42.378789 8508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:42.383999 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 8508
I20260927 22:57:42.384217 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:42.384316 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:42.385171 4394 external_mini_cluster.cc:1428] Setting key 75ebffb5c51f0ae02973f2a3784c8aef
I20260927 22:57:42.400431 8574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe: Bootstrap starting.
Sep 27 22:57:42 dist-test-slave-5pd3 krb5kdc[8495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:42.404847 8574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:42.406432 8574 log.cc:824] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:42.410537 8574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe: No bootstrap required, opened a new log
I20260927 22:57:42.413322 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:42.388089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34090 (local address 127.4.74.190:41043)
0927 22:57:42.389228 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:42.389245 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:42.389295 (+ 50us) server_negotiation.cc:407] Connection header received
0927 22:57:42.392241 (+ 2946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:42.392278 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:42.392892 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:42.393384 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:42.395219 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:42.397001 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:42.398526 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:42.399347 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:42.399702 (+ 355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:42.404592 (+ 4890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:42.404601 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:42.404623 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:42.404672 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:42.408809 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:42.409606 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:42.409614 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:42.409621 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:42.409798 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:42.410164 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:42.410170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:42.410173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:42.410853 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:42.411208 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:42.411687 (+ 479us) server_negotiation.cc:299] Negotiation successful
0927 22:57:42.412173 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":494,"threads_started":1}
I20260927 22:57:42.420490 8574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3816d958694d779a0e9a6e044437fe" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 41043 } }
I20260927 22:57:42.421239 8574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:42.421392 8574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a3816d958694d779a0e9a6e044437fe, State: Initialized, Role: FOLLOWER
I20260927 22:57:42.422255 8574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3816d958694d779a0e9a6e044437fe" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 41043 } }
I20260927 22:57:42.422566 8574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:42.422755 8574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:42.422966 8574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:42.425876 8574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3816d958694d779a0e9a6e044437fe" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 41043 } }
I20260927 22:57:42.426483 8574 leader_election.cc:302] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a3816d958694d779a0e9a6e044437fe; no voters:
I20260927 22:57:42.427140 8574 leader_election.cc:288] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:42.427362 8579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:42.429150 8579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [term 1 LEADER]: Becoming Leader. State: Replica: 4a3816d958694d779a0e9a6e044437fe, State: Running, Role: LEADER
I20260927 22:57:42.430008 8579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3816d958694d779a0e9a6e044437fe" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 41043 } }
I20260927 22:57:42.430267 8574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:42.435308 8580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a3816d958694d779a0e9a6e044437fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3816d958694d779a0e9a6e044437fe" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 41043 } } }
I20260927 22:57:42.435932 8580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:42.436620 8581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a3816d958694d779a0e9a6e044437fe. Latest consensus state: current_term: 1 leader_uuid: "4a3816d958694d779a0e9a6e044437fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a3816d958694d779a0e9a6e044437fe" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 41043 } } }
I20260927 22:57:42.437532 8581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:42.438311 8588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:42.445094 8588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:42.459785 8588 catalog_manager.cc:1347] Generated new cluster ID: 24fb89fa1af244de94e73293bd308af1
I20260927 22:57:42.459954 8588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:42.473264 8588 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:42.475044 8588 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:42.487968 8588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe: Generated new TSK 0
I20260927 22:57:42.489491 8588 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:57:42.561115 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:41043
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:43.000187 8600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:43.000597 8600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:43.000784 8600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:43.010221 8600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:43.010421 8600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:43.010707 8600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:57:43.023619 8600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:41043
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:43.025527 8600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:43.027438 8600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:43.044868 8605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:43.045050 8606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:43.045053 8608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:43.046118 8600 server_base.cc:1034] running on GCE node
I20260927 22:57:43.046962 8600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:43.048352 8600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:43.049759 8600 hybrid_clock.cc:648] HybridClock initialized: now 1790549863049686 us; error 63 us; skew 500 ppm
I20260927 22:57:43.050341 8600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:43 dist-test-slave-5pd3 krb5kdc[8495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549863, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:43.067170 8600 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:57:43.068989 8600 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:43.069169 8600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:43.069602 8600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:43.073477 8600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "e507ffa5b70e4fd584299bbdd3b3262f"
format_stamp: "Formatted at 2026-09-27 22:57:43 on dist-test-slave-5pd3"
server_key: "3f6d0ec2f0a352cae0b98a368b29e86e"
server_key_iv: "a6a8537be6d6d383555c7f84cb0d1a13"
server_key_version: "encryptionkey@0"
I20260927 22:57:43.074457 8600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "e507ffa5b70e4fd584299bbdd3b3262f"
format_stamp: "Formatted at 2026-09-27 22:57:43 on dist-test-slave-5pd3"
server_key: "3f6d0ec2f0a352cae0b98a368b29e86e"
server_key_iv: "a6a8537be6d6d383555c7f84cb0d1a13"
server_key_version: "encryptionkey@0"
I20260927 22:57:43.080961 8600 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260927 22:57:43.085556 8614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:43.087733 8600 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260927 22:57:43.087975 8600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "e507ffa5b70e4fd584299bbdd3b3262f"
format_stamp: "Formatted at 2026-09-27 22:57:43 on dist-test-slave-5pd3"
server_key: "3f6d0ec2f0a352cae0b98a368b29e86e"
server_key_iv: "a6a8537be6d6d383555c7f84cb0d1a13"
server_key_version: "encryptionkey@0"
I20260927 22:57:43.088274 8600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:43.108489 8600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:43.114158 8600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:43.114526 8600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:43.115823 8600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:43.117563 8600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:43.117717 8600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:43.117866 8600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:43.117961 8600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:43.158473 8600 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:38721
I20260927 22:57:43.158538 8726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:38721 every 8 connection(s)
I20260927 22:57:43.159678 8600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:57:43.169845 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 8600
I20260927 22:57:43.170157 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:57:43.170919 4394 external_mini_cluster.cc:1428] Setting key 154724e8da8978e0ca93a01ca103c244
Sep 27 22:57:43 dist-test-slave-5pd3 krb5kdc[8495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549863, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:43.181820 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:43.162902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:55433 (local address 127.4.74.190:41043)
0927 22:57:43.163402 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:43.163407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:43.164735 (+ 1328us) server_negotiation.cc:407] Connection header received
0927 22:57:43.166733 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:43.166739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:43.166836 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:43.166966 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:43.168709 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:43.169497 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:43.171254 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:43.171713 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:43.171848 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:43.175647 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:43.175653 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:43.175656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:43.175698 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:43.178520 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:43.179359 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:43.179366 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:43.179375 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:43.179491 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:43.179973 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:43.179978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:43.179981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:43.180258 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:43.180418 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:43.181196 (+ 778us) server_negotiation.cc:299] Negotiation successful
0927 22:57:43.181425 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":293,"threads_started":1}
I20260927 22:57:43.183233 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:43.163537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:41043 (local address 127.4.74.129:55433)
0927 22:57:43.164541 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:43.164594 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:43.166437 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:43.167220 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:43.167229 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:43.167890 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:43.168529 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:43.168546 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:43.169654 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:43.169660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:43.171110 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:43.171122 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:43.171945 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:43.171954 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:43.172197 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:43.173049 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:43.173075 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:43.175337 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:43.178701 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:43.178711 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:43.178738 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:43.179218 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:43.179633 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:43.179641 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:43.179647 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:43.179836 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:43.180419 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:43.180429 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:43.180807 (+ 378us) client_negotiation.cc:769] Sending connection context
0927 22:57:43.181099 (+ 292us) client_negotiation.cc:240] Negotiation successful
0927 22:57:43.181685 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":380,"threads_started":1}
I20260927 22:57:43.185472 8727 heartbeater.cc:347] Connected to a master server at 127.4.74.190:41043
I20260927 22:57:43.185863 8727 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:43.186779 8727 heartbeater.cc:511] Master 127.4.74.190:41043 requested a full tablet report, sending...
I20260927 22:57:43.190078 8539 ts_manager.cc:194] Registered new tserver with Master: e507ffa5b70e4fd584299bbdd3b3262f (127.4.74.129:38721)
I20260927 22:57:43.192477 8539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:55433
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:57:43.224066 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:41043
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:43.581429 8733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:43.581729 8733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:43.581897 8733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:43.589372 8733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:43.589497 8733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:43.589699 8733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:57:43.600136 8733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:41043
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:43.601562 8733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:43.602939 8733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:43.618774 8741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:43.618718 8738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:43.618830 8739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:43.620301 8733 server_base.cc:1034] running on GCE node
I20260927 22:57:43.621091 8733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:43.622362 8733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:43.623662 8733 hybrid_clock.cc:648] HybridClock initialized: now 1790549863623597 us; error 59 us; skew 500 ppm
I20260927 22:57:43.624050 8733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:43 dist-test-slave-5pd3 krb5kdc[8495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549863, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:43.628974 8733 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:57:43.637497 8733 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:43.637661 8733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:43.638094 8733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:43.641825 8733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "ac1299abd8ac4c919388704bae694301"
format_stamp: "Formatted at 2026-09-27 22:57:43 on dist-test-slave-5pd3"
server_key: "b63437c2ef0bf08f48b5527f429faed4"
server_key_iv: "c206d2e1b423fa8de03c4a1006d3e387"
server_key_version: "encryptionkey@0"
I20260927 22:57:43.642654 8733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "ac1299abd8ac4c919388704bae694301"
format_stamp: "Formatted at 2026-09-27 22:57:43 on dist-test-slave-5pd3"
server_key: "b63437c2ef0bf08f48b5527f429faed4"
server_key_iv: "c206d2e1b423fa8de03c4a1006d3e387"
server_key_version: "encryptionkey@0"
I20260927 22:57:43.648499 8733 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260927 22:57:43.652802 8747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:43.654623 8733 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:57:43.654881 8733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "ac1299abd8ac4c919388704bae694301"
format_stamp: "Formatted at 2026-09-27 22:57:43 on dist-test-slave-5pd3"
server_key: "b63437c2ef0bf08f48b5527f429faed4"
server_key_iv: "c206d2e1b423fa8de03c4a1006d3e387"
server_key_version: "encryptionkey@0"
I20260927 22:57:43.655145 8733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:43.671689 8733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:43.676438 8733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:43.676789 8733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:43.677979 8733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:43.679528 8733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:43.679709 8733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260927 22:57:43.679863 8733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:43.679955 8733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:43.720938 8733 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:39819
I20260927 22:57:43.721153 8859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:39819 every 8 connection(s)
I20260927 22:57:43.721922 8733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:57:43.728113 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 8733
I20260927 22:57:43.728382 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:57:43.728921 4394 external_mini_cluster.cc:1428] Setting key 9c1e1de8c521daa5629f785568b584fe
Sep 27 22:57:43 dist-test-slave-5pd3 krb5kdc[8495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549863, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:43.746291 8862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:43.725307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:60969 (local address 127.4.74.190:41043)
0927 22:57:43.725895 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:43.725900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:43.727128 (+ 1228us) server_negotiation.cc:407] Connection header received
0927 22:57:43.729320 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:43.729326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:43.729434 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:43.729558 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:43.731518 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:43.732376 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:43.733970 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:43.734410 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:43.734543 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:43.738494 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:43.738499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:43.738502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:43.738533 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:43.742017 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:43.743143 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:43.743152 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:43.743158 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:43.743331 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:43.743900 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:43.743907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:43.743913 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:43.744334 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:43.744499 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:43.745489 (+ 990us) server_negotiation.cc:299] Negotiation successful
0927 22:57:43.745771 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":306,"threads_started":1}
I20260927 22:57:43.747623 8863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:43.725981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:41043 (local address 127.4.74.130:60969)
0927 22:57:43.726879 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:43.726964 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:43.728996 (+ 2032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:43.729824 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:43.729834 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:43.730574 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:43.731320 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:43.731338 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:43.732530 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:43.732537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:43.733806 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:43.733819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:43.734658 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:43.734666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:43.734893 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:43.735774 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:43.735801 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:43.738243 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:43.742280 (+ 4037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:43.742291 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:43.742335 (+ 44us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:43.742971 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:43.743472 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:43.743476 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:43.743479 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:43.743765 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:43.744517 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:43.744525 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:43.745000 (+ 475us) client_negotiation.cc:769] Sending connection context
0927 22:57:43.745339 (+ 339us) client_negotiation.cc:240] Negotiation successful
0927 22:57:43.745933 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":365,"threads_started":1}
I20260927 22:57:43.749800 8860 heartbeater.cc:347] Connected to a master server at 127.4.74.190:41043
I20260927 22:57:43.750273 8860 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 22:57:43.751271 8860 heartbeater.cc:511] Master 127.4.74.190:41043 requested a full tablet report, sending...
I20260927 22:57:43.753309 8539 ts_manager.cc:194] Registered new tserver with Master: ac1299abd8ac4c919388704bae694301 (127.4.74.130:39819)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
I20260927 22:57:43.754455 8539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:60969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:57:43.784215 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:41043
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:44.134557 8866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:44.134929 8866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:44.135102 8866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:44.142735 8866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:44.142911 8866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:44.143100 8866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:57:44.153940 8866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:41043
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:44.155387 8866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:44.156873 8866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:44.173123 8871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:44.173273 8874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:44.173310 8872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:44.174923 8866 server_base.cc:1034] running on GCE node
I20260927 22:57:44.175712 8866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:44.176755 8866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:44.178025 8866 hybrid_clock.cc:648] HybridClock initialized: now 1790549864177959 us; error 46 us; skew 500 ppm
I20260927 22:57:44.178428 8866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:44 dist-test-slave-5pd3 krb5kdc[8495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549864, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:44.181984 8866 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:57:44.183261 8866 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:44.183362 8866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:44.183724 8866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:44.186672 8866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "7e73ea8c6b784fe6a149418780050b3e"
format_stamp: "Formatted at 2026-09-27 22:57:44 on dist-test-slave-5pd3"
server_key: "1a53986886bf2358bafbceec4bc4a14b"
server_key_iv: "7a7ee1485ac0d77b77d8de01a1d6c737"
server_key_version: "encryptionkey@0"
I20260927 22:57:44.187474 8866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "7e73ea8c6b784fe6a149418780050b3e"
format_stamp: "Formatted at 2026-09-27 22:57:44 on dist-test-slave-5pd3"
server_key: "1a53986886bf2358bafbceec4bc4a14b"
server_key_iv: "7a7ee1485ac0d77b77d8de01a1d6c737"
server_key_version: "encryptionkey@0"
I20260927 22:57:44.193434 8866 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260927 22:57:44.197793 8727 heartbeater.cc:503] Master 127.4.74.190:41043 was elected leader, sending a full tablet report...
I20260927 22:57:44.197933 8880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:44.200174 8866 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 22:57:44.200402 8866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "7e73ea8c6b784fe6a149418780050b3e"
format_stamp: "Formatted at 2026-09-27 22:57:44 on dist-test-slave-5pd3"
server_key: "1a53986886bf2358bafbceec4bc4a14b"
server_key_iv: "7a7ee1485ac0d77b77d8de01a1d6c737"
server_key_version: "encryptionkey@0"
I20260927 22:57:44.200659 8866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:44.221740 8866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:44.226155 8866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:44.226462 8866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:44.227638 8866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:44.229190 8866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:44.229316 8866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:44.229460 8866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:44.229548 8866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:44.268666 8866 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:37063
I20260927 22:57:44.268764 8992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:37063 every 8 connection(s)
I20260927 22:57:44.269659 8866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:57:44.269922 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 8866
I20260927 22:57:44.270169 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:57:44.270862 4394 external_mini_cluster.cc:1428] Setting key 3079b242ac95097290d1e4c661ee8b61
Sep 27 22:57:44 dist-test-slave-5pd3 krb5kdc[8495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549864, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:44.293840 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.274118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:50477 (local address 127.4.74.190:41043)
0927 22:57:44.274776 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.274786 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.275809 (+ 1023us) server_negotiation.cc:407] Connection header received
0927 22:57:44.277812 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.277817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.277905 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.278022 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:44.279917 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.280863 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.282671 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.283214 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.283377 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.287406 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:44.287412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:44.287416 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:44.287462 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:44.290457 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:44.291277 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:44.291282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:44.291284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:44.291399 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:44.291862 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:44.291867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:44.291870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:44.292152 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:44.292373 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.293196 (+ 823us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.293465 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":327,"threads_started":1}
I20260927 22:57:44.294803 8996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.274515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:41043 (local address 127.4.74.131:50477)
0927 22:57:44.275593 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:44.275655 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:44.277503 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:44.278282 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:44.278291 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:44.278895 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:44.279629 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.279668 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.281041 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.281046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.282451 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.282465 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.283474 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.283480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.283798 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.284600 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:44.284630 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:44.287112 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:44.290631 (+ 3519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:44.290642 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:44.290662 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:44.291134 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:44.291514 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:44.291519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:44.291523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:44.291742 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:44.292354 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:44.292363 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:44.292718 (+ 355us) client_negotiation.cc:769] Sending connection context
0927 22:57:44.293000 (+ 282us) client_negotiation.cc:240] Negotiation successful
0927 22:57:44.293555 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":457,"threads_started":1}
I20260927 22:57:44.296581 8993 heartbeater.cc:347] Connected to a master server at 127.4.74.190:41043
I20260927 22:57:44.296991 8993 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:44.297835 8993 heartbeater.cc:511] Master 127.4.74.190:41043 requested a full tablet report, sending...
I20260927 22:57:44.299533 8539 ts_manager.cc:194] Registered new tserver with Master: 7e73ea8c6b784fe6a149418780050b3e (127.4.74.131:37063)
I20260927 22:57:44.300738 8539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:50477
I20260927 22:57:44.302578 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 22:57:44.322961 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.308861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34098 (local address 127.4.74.190:41043)
0927 22:57:44.309091 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.309096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.309744 (+ 648us) server_negotiation.cc:407] Connection header received
0927 22:57:44.309982 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.309987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.310075 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.310191 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:44.311854 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.312657 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.313971 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.314491 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.314598 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.317182 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:44.317186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:44.317190 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:44.317226 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:44.320097 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:44.320862 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:44.320866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:44.320868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:44.320999 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:44.321444 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:44.321447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:44.321449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:44.321885 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:44.322078 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.322476 (+ 398us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.322675 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260927 22:57:44.328229 8539 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:57:44.336386 8539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:44.379861 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.351881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57212 (local address 127.4.74.130:39819)
0927 22:57:44.365675 (+ 13794us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.365690 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.366673 (+ 983us) server_negotiation.cc:407] Connection header received
0927 22:57:44.367117 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.367139 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.367407 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.367622 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:44.369076 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.371365 (+ 2289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.376422 (+ 5057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.378053 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.378160 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.378365 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.379242 (+ 877us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.379445 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13523,"thread_start_us":13344,"threads_started":1}
I20260927 22:57:44.379992 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.352043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:37063 (local address 127.0.0.1:49080)
0927 22:57:44.366533 (+ 14490us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:44.366555 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:44.366797 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:44.368112 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:44.368120 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:44.368159 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:44.370778 (+ 2619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.370801 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.373430 (+ 2629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.373439 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.376110 (+ 2671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.376124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.378622 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.378628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.378946 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.378983 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 22:57:44.379149 (+ 166us) client_negotiation.cc:240] Negotiation successful
0927 22:57:44.379427 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14329,"mutex_wait_us":92,"thread_start_us":14797,"threads_started":1}
I20260927 22:57:44.380841 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.351640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:39819 (local address 127.0.0.1:57212)
0927 22:57:44.366449 (+ 14809us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:44.366503 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:44.366829 (+ 326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:44.368062 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:44.368073 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:44.368117 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:44.368782 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.368805 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.371651 (+ 2846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.371660 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.376212 (+ 4552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.376226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.378584 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.378590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.378873 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.378982 (+ 109us) client_negotiation.cc:769] Sending connection context
0927 22:57:44.380234 (+ 1252us) client_negotiation.cc:240] Negotiation successful
0927 22:57:44.380377 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14513,"mutex_wait_us":1265,"thread_start_us":14145,"threads_started":1}
I20260927 22:57:44.381050 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.352642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.4.74.131:37063)
0927 22:57:44.363213 (+ 10571us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.363226 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.366809 (+ 3583us) server_negotiation.cc:407] Connection header received
0927 22:57:44.367136 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.367146 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.367498 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.367750 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:44.371088 (+ 3338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.373188 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.376337 (+ 3149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.378035 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.378179 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.378450 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.380377 (+ 1927us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.380609 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10253,"thread_start_us":10030,"threads_started":1}
I20260927 22:57:44.385015 8795 tablet_service.cc:1467] Processing CreateTablet for tablet 31eb02a0c6bd499d96bea3b11134bc05 (DEFAULT_TABLE table=test-table [id=90800fd0b7324fcc8b35acfeb257d654]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:44.386493 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.367700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36554 (local address 127.4.74.129:38721)
0927 22:57:44.374984 (+ 7284us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.374997 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.375815 (+ 818us) server_negotiation.cc:407] Connection header received
0927 22:57:44.375916 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.375926 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.376242 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.376448 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:44.377335 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.378849 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.382687 (+ 3838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.384334 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.384504 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.384765 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.385897 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.386110 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6997,"thread_start_us":6773,"threads_started":1}
I20260927 22:57:44.386549 8795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31eb02a0c6bd499d96bea3b11134bc05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:44.387194 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.367452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:38721 (local address 127.0.0.1:36554)
0927 22:57:44.375499 (+ 8047us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:44.375524 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:44.375697 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:44.376707 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:44.376711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:44.376737 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:44.377143 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.377155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.379007 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.379012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.382465 (+ 3453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.382481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.384943 (+ 2462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.384950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.385152 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.385188 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 22:57:44.385301 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 22:57:44.385482 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7906,"thread_start_us":7766,"threads_started":1}
I20260927 22:57:44.387346 8928 tablet_service.cc:1467] Processing CreateTablet for tablet 31eb02a0c6bd499d96bea3b11134bc05 (DEFAULT_TABLE table=test-table [id=90800fd0b7324fcc8b35acfeb257d654]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:44.389083 8928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31eb02a0c6bd499d96bea3b11134bc05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:44.397435 8662 tablet_service.cc:1467] Processing CreateTablet for tablet 31eb02a0c6bd499d96bea3b11134bc05 (DEFAULT_TABLE table=test-table [id=90800fd0b7324fcc8b35acfeb257d654]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:44.399461 8662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31eb02a0c6bd499d96bea3b11134bc05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:44.402560 9012 tablet_bootstrap.cc:492] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: Bootstrap starting.
I20260927 22:57:44.406536 9012 tablet_bootstrap.cc:654] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:44.407909 9012 log.cc:824] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:44.409006 9013 tablet_bootstrap.cc:492] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: Bootstrap starting.
I20260927 22:57:44.414099 9012 tablet_bootstrap.cc:492] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: No bootstrap required, opened a new log
I20260927 22:57:44.414243 9013 tablet_bootstrap.cc:654] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:44.414433 9012 ts_tablet_manager.cc:1397] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260927 22:57:44.415774 9013 log.cc:824] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:44.418424 9016 tablet_bootstrap.cc:492] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: Bootstrap starting.
I20260927 22:57:44.419777 9013 tablet_bootstrap.cc:492] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: No bootstrap required, opened a new log
I20260927 22:57:44.420146 9013 ts_tablet_manager.cc:1397] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260927 22:57:44.422772 9016 tablet_bootstrap.cc:654] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:44.423221 9012 raft_consensus.cc:348] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1299abd8ac4c919388704bae694301" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39819 } } peers { permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 } } peers { permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 } }
I20260927 22:57:44.423672 9012 raft_consensus.cc:374] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:44.423774 9012 raft_consensus.cc:729] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1299abd8ac4c919388704bae694301, State: Initialized, Role: FOLLOWER
I20260927 22:57:44.424315 9016 log.cc:824] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:44.424397 9012 consensus_queue.cc:260] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1299abd8ac4c919388704bae694301" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39819 } } peers { permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 } } peers { permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 } }
I20260927 22:57:44.425661 8860 heartbeater.cc:503] Master 127.4.74.190:41043 was elected leader, sending a full tablet report...
I20260927 22:57:44.426350 9012 ts_tablet_manager.cc:1428] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260927 22:57:44.429556 9016 tablet_bootstrap.cc:492] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: No bootstrap required, opened a new log
I20260927 22:57:44.429875 9016 ts_tablet_manager.cc:1397] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260927 22:57:44.432263 9013 raft_consensus.cc:348] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1299abd8ac4c919388704bae694301" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39819 } } peers { permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 } } peers { permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 } }
I20260927 22:57:44.432807 9013 raft_consensus.cc:374] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:44.432938 9013 raft_consensus.cc:729] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e73ea8c6b784fe6a149418780050b3e, State: Initialized, Role: FOLLOWER
I20260927 22:57:44.433713 9013 consensus_queue.cc:260] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1299abd8ac4c919388704bae694301" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39819 } } peers { permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 } } peers { permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 } }
I20260927 22:57:44.435093 8993 heartbeater.cc:503] Master 127.4.74.190:41043 was elected leader, sending a full tablet report...
I20260927 22:57:44.435827 9013 ts_tablet_manager.cc:1428] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260927 22:57:44.439972 9016 raft_consensus.cc:348] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1299abd8ac4c919388704bae694301" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39819 } } peers { permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 } } peers { permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 } }
I20260927 22:57:44.440549 9016 raft_consensus.cc:374] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:44.440655 9016 raft_consensus.cc:729] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e507ffa5b70e4fd584299bbdd3b3262f, State: Initialized, Role: FOLLOWER
I20260927 22:57:44.441365 9016 consensus_queue.cc:260] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1299abd8ac4c919388704bae694301" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39819 } } peers { permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 } } peers { permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 } }
I20260927 22:57:44.443285 9016 ts_tablet_manager.cc:1428] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260927 22:57:44.474390 8861 tablet.cc:2369] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:57:44.483935 9017 raft_consensus.cc:482] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:44.484212 9017 raft_consensus.cc:504] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1299abd8ac4c919388704bae694301" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39819 } } peers { permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 } } peers { permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 } }
I20260927 22:57:44.485984 9017 leader_election.cc:288] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e73ea8c6b784fe6a149418780050b3e (127.4.74.131:37063), e507ffa5b70e4fd584299bbdd3b3262f (127.4.74.129:38721)
I20260927 22:57:44.495992 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.486495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:35265 (local address 127.4.74.131:37063)
0927 22:57:44.486975 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.486983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.487638 (+ 655us) server_negotiation.cc:407] Connection header received
0927 22:57:44.487998 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.488004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.488097 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.488172 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:44.489117 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.490188 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.493319 (+ 3131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.494654 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.494843 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.494921 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.495582 (+ 661us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.495717 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260927 22:57:44.496040 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.487878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:56741 (local address 127.4.74.129:38721)
0927 22:57:44.488094 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.488101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.488574 (+ 473us) server_negotiation.cc:407] Connection header received
0927 22:57:44.488668 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.488672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.488744 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.488860 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:57:44.489515 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.490506 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.493319 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.494640 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.494839 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.494921 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.495577 (+ 656us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.495717 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 22:57:44.496292 9022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.487481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:38721 (local address 127.4.74.130:56741)
0927 22:57:44.488402 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:44.488422 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:44.488552 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:44.488921 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:44.488926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:44.488949 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:44.489342 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.489350 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.490646 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.490651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.493089 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.493107 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.495018 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.495026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.495264 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.495385 (+ 121us) client_negotiation.cc:769] Sending connection context
0927 22:57:44.495500 (+ 115us) client_negotiation.cc:240] Negotiation successful
0927 22:57:44.495733 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":160,"thread_start_us":360,"threads_started":1}
I20260927 22:57:44.496747 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.486388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:37063 (local address 127.4.74.130:35265)
0927 22:57:44.487459 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:44.487488 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:44.487764 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:44.488446 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:44.488453 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:44.488487 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:57:44.488929 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.488941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.490383 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.490392 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.493089 (+ 2697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:44.493107 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.495029 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.495034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:44.495266 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.495382 (+ 116us) client_negotiation.cc:769] Sending connection context
0927 22:57:44.495500 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 22:57:44.495733 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"mutex_wait_us":102,"thread_start_us":629,"threads_started":1}
I20260927 22:57:44.497767 8682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31eb02a0c6bd499d96bea3b11134bc05" candidate_uuid: "ac1299abd8ac4c919388704bae694301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" is_pre_election: true
I20260927 22:57:44.497965 8948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31eb02a0c6bd499d96bea3b11134bc05" candidate_uuid: "ac1299abd8ac4c919388704bae694301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e73ea8c6b784fe6a149418780050b3e" is_pre_election: true
I20260927 22:57:44.498430 8682 raft_consensus.cc:2454] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac1299abd8ac4c919388704bae694301 in term 0.
I20260927 22:57:44.498659 8948 raft_consensus.cc:2454] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac1299abd8ac4c919388704bae694301 in term 0.
I20260927 22:57:44.499478 8751 leader_election.cc:302] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac1299abd8ac4c919388704bae694301, e507ffa5b70e4fd584299bbdd3b3262f; no voters:
I20260927 22:57:44.500007 9017 raft_consensus.cc:2781] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:44.500209 9017 raft_consensus.cc:482] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:44.500352 9017 raft_consensus.cc:3037] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:44.502566 9017 raft_consensus.cc:504] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1299abd8ac4c919388704bae694301" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39819 } } peers { permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 } } peers { permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 } }
I20260927 22:57:44.503233 9017 leader_election.cc:288] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [CANDIDATE]: Term 1 election: Requested vote from peers 7e73ea8c6b784fe6a149418780050b3e (127.4.74.131:37063), e507ffa5b70e4fd584299bbdd3b3262f (127.4.74.129:38721)
I20260927 22:57:44.504148 8682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31eb02a0c6bd499d96bea3b11134bc05" candidate_uuid: "ac1299abd8ac4c919388704bae694301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e507ffa5b70e4fd584299bbdd3b3262f"
I20260927 22:57:44.504149 8948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31eb02a0c6bd499d96bea3b11134bc05" candidate_uuid: "ac1299abd8ac4c919388704bae694301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e73ea8c6b784fe6a149418780050b3e"
I20260927 22:57:44.504479 8682 raft_consensus.cc:3037] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:44.504480 8948 raft_consensus.cc:3037] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:44.508322 8948 raft_consensus.cc:2454] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac1299abd8ac4c919388704bae694301 in term 1.
I20260927 22:57:44.508322 8682 raft_consensus.cc:2454] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac1299abd8ac4c919388704bae694301 in term 1.
I20260927 22:57:44.508980 8751 leader_election.cc:302] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac1299abd8ac4c919388704bae694301, e507ffa5b70e4fd584299bbdd3b3262f; no voters:
I20260927 22:57:44.509351 9017 raft_consensus.cc:2781] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:44.509972 9017 raft_consensus.cc:686] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 1 LEADER]: Becoming Leader. State: Replica: ac1299abd8ac4c919388704bae694301, State: Running, Role: LEADER
I20260927 22:57:44.510676 9017 consensus_queue.cc:237] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1299abd8ac4c919388704bae694301" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39819 } } peers { permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 } } peers { permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 } }
I20260927 22:57:44.515655 8538 catalog_manager.cc:5495] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac1299abd8ac4c919388704bae694301 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "ac1299abd8ac4c919388704bae694301" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1299abd8ac4c919388704bae694301" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 39819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 } health_report { overall_health: UNKNOWN } } }
W20260927 22:57:44.521831 8994 tablet.cc:2369] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:57:44.566458 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.560387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.4.74.130:39819)
0927 22:57:44.560668 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.560678 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.560702 (+ 24us) server_negotiation.cc:407] Connection header received
0927 22:57:44.560855 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.560860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.560958 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.561154 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:57:44.561564 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.562342 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.563839 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.564296 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.564443 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.565001 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:57:44.565737 (+ 736us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:57:44.565958 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.566011 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.566108 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260927 22:57:44.577346 8682 raft_consensus.cc:1261] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f [term 1 FOLLOWER]: Refusing update from remote peer ac1299abd8ac4c919388704bae694301: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:44.577345 8948 raft_consensus.cc:1261] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e [term 1 FOLLOWER]: Refusing update from remote peer ac1299abd8ac4c919388704bae694301: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:44.578814 9017 consensus_queue.cc:1035] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e507ffa5b70e4fd584299bbdd3b3262f" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:44.579209 9023 consensus_queue.cc:1035] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e73ea8c6b784fe6a149418780050b3e" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:44.593935 9026 mvcc.cc:205] Tried to move back new op lower bound from 7334092245292908544 to 7334092245040562176. Current Snapshot: MvccSnapshot[applied={T|T < 7334092245292908544}]
I20260927 22:57:44.618067 8538 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:44.618425 8538 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:44.622880 8538 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a3816d958694d779a0e9a6e044437fe: Sending DeleteTablet for 3 replicas of tablet 31eb02a0c6bd499d96bea3b11134bc05
I20260927 22:57:44.624528 8795 tablet_service.cc:1514] Processing DeleteTablet for tablet 31eb02a0c6bd499d96bea3b11134bc05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:44 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:57212
I20260927 22:57:44.624619 8662 tablet_service.cc:1514] Processing DeleteTablet for tablet 31eb02a0c6bd499d96bea3b11134bc05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:44 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:36554
I20260927 22:57:44.625353 8928 tablet_service.cc:1514] Processing DeleteTablet for tablet 31eb02a0c6bd499d96bea3b11134bc05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:44 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:49080
I20260927 22:57:44.625916 9036 tablet_replica.cc:330] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: stopping tablet replica
I20260927 22:57:44.626706 9036 raft_consensus.cc:2229] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:57:44.627857 9036 raft_consensus.cc:2258] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:44.627923 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 8600
I20260927 22:57:44.633271 9036 ts_tablet_manager.cc:1905] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:44.646162 9036 ts_tablet_manager.cc:1918] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:57:44.646695 9036 log.cc:1196] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/31eb02a0c6bd499d96bea3b11134bc05
I20260927 22:57:44.647754 9036 ts_tablet_manager.cc:1939] T 31eb02a0c6bd499d96bea3b11134bc05 P ac1299abd8ac4c919388704bae694301: Deleting consensus metadata
I20260927 22:57:44.651132 8523 catalog_manager.cc:4841] TS ac1299abd8ac4c919388704bae694301 (127.4.74.130:39819): tablet 31eb02a0c6bd499d96bea3b11134bc05 (table test-table [id=90800fd0b7324fcc8b35acfeb257d654]) successfully deleted
I20260927 22:57:44.653237 9038 tablet_replica.cc:330] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: stopping tablet replica
I20260927 22:57:44.653806 9038 raft_consensus.cc:2229] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:44.654412 9038 raft_consensus.cc:2258] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:44.657032 9038 ts_tablet_manager.cc:1905] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:44.659695 9037 tablet_replica.cc:330] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: stopping tablet replica
I20260927 22:57:44.660245 9037 raft_consensus.cc:2229] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:44.660856 9037 raft_consensus.cc:2258] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:44.665019 9037 ts_tablet_manager.cc:1905] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 27 22:57:44 dist-test-slave-5pd3 krb5kdc[8495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:44.667882 9038 ts_tablet_manager.cc:1918] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:57:44.668097 9038 log.cc:1196] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/31eb02a0c6bd499d96bea3b11134bc05
I20260927 22:57:44.668704 9038 ts_tablet_manager.cc:1939] T 31eb02a0c6bd499d96bea3b11134bc05 P 7e73ea8c6b784fe6a149418780050b3e: Deleting consensus metadata
I20260927 22:57:44.670899 8526 catalog_manager.cc:4841] TS 7e73ea8c6b784fe6a149418780050b3e (127.4.74.131:37063): tablet 31eb02a0c6bd499d96bea3b11134bc05 (table test-table [id=90800fd0b7324fcc8b35acfeb257d654]) successfully deleted
I20260927 22:57:44.674907 9037 ts_tablet_manager.cc:1918] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:57:44.675191 9037 log.cc:1196] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/31eb02a0c6bd499d96bea3b11134bc05
I20260927 22:57:44.675868 9037 ts_tablet_manager.cc:1939] T 31eb02a0c6bd499d96bea3b11134bc05 P e507ffa5b70e4fd584299bbdd3b3262f: Deleting consensus metadata
I20260927 22:57:44.676980 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.629985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36556 (local address 127.4.74.129:38721)
0927 22:57:44.630253 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.630260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.652326 (+ 22066us) server_negotiation.cc:407] Connection header received
0927 22:57:44.652561 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.652572 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.652778 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.652939 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:44.656369 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.658187 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.660696 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.661456 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.661612 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.668109 (+ 6497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:44.668128 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:44.668151 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:44.668211 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:44.672931 (+ 4720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:44.674176 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:44.674188 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:44.674192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:44.674415 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:44.674947 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:44.674952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:44.674955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:44.675535 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:44.675957 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.676321 (+ 364us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.676562 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260927 22:57:44.677922 8722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:44.678572 8526 catalog_manager.cc:4841] TS e507ffa5b70e4fd584299bbdd3b3262f (127.4.74.129:38721): tablet 31eb02a0c6bd499d96bea3b11134bc05 (table test-table [id=90800fd0b7324fcc8b35acfeb257d654]) successfully deleted
I20260927 22:57:44.781795 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 8600
I20260927 22:57:44.801379 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 8733
Sep 27 22:57:44 dist-test-slave-5pd3 krb5kdc[8495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:57:44.822016 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.802774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57230 (local address 127.4.74.130:39819)
0927 22:57:44.803036 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.803044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.803069 (+ 25us) server_negotiation.cc:407] Connection header received
0927 22:57:44.803151 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.803159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.803234 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.803685 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:44.805402 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.806517 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.807954 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.808457 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.808611 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.814286 (+ 5675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:44.814306 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:44.814345 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:44.814447 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:44.818138 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:44.819313 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:44.819339 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:44.819342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:44.819539 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:44.820108 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:44.820113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:44.820115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:44.820636 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:44.821093 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.821309 (+ 216us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.821603 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260927 22:57:44.823151 8855 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:44.962608 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 8733
I20260927 22:57:44.980192 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 8866
Sep 27 22:57:44 dist-test-slave-5pd3 krb5kdc[8495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:57:44.997539 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:44.981268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49082 (local address 127.4.74.131:37063)
0927 22:57:44.981535 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:44.981541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:44.981565 (+ 24us) server_negotiation.cc:407] Connection header received
0927 22:57:44.981780 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:44.981786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:44.981884 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:44.982001 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:44.983348 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.984315 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.985579 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:44.985980 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:44.986117 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:44.991216 (+ 5099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:44.991234 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:44.991251 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:44.991294 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:44.994238 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:44.995011 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:44.995019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:44.995021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:44.995169 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:44.995754 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:44.995762 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:44.995767 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:44.996216 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:44.996577 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:44.996911 (+ 334us) server_negotiation.cc:299] Negotiation successful
0927 22:57:44.997171 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260927 22:57:44.998505 8988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:45.099886 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 8866
I20260927 22:57:45.117894 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 8508
I20260927 22:57:45.119344 8569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:45.231243 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 8508
2026-09-27T22:57:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4771 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-27T22:57:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:45Z Disabled control of system clock
I20260927 22:57:45.307431 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:40613
--webserver_interface=127.4.74.190
--webserver_port=0
--builtin_ntp_servers=127.4.74.148:33567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:40613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 22:57:45.639653 9047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:45.640053 9047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:45.640143 9047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:45.647610 9047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:45.647745 9047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:45.647784 9047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:45.647837 9047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:45.647891 9047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:45.658241 9047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:33567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:40613
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:45.659704 9047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:45.661500 9047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:45.674777 9055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:45.674823 9052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:45.674887 9053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:45.679735 9047 server_base.cc:1034] running on GCE node
I20260927 22:57:45.680761 9047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:45.682351 9047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:45.683633 9047 hybrid_clock.cc:648] HybridClock initialized: now 1790549865683576 us; error 53 us; skew 500 ppm
I20260927 22:57:45.683988 9047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:57:45.686524 9047 webserver.cc:460] Webserver started at http://127.4.74.190:46703/ using document root <none> and password file <none>
I20260927 22:57:45.687255 9047 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:45.687350 9047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:45.687696 9047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:45.690644 9047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "a51a175f120341d08400fcae9eeafedb"
format_stamp: "Formatted at 2026-09-27 22:57:45 on dist-test-slave-5pd3"
server_key: "c654b5d3504a3f8b56569f739270b979"
server_key_iv: "07c12d758307b4f394fa2fe5fa4d36bd"
server_key_version: "encryptionkey@0"
I20260927 22:57:45.691417 9047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "a51a175f120341d08400fcae9eeafedb"
format_stamp: "Formatted at 2026-09-27 22:57:45 on dist-test-slave-5pd3"
server_key: "c654b5d3504a3f8b56569f739270b979"
server_key_iv: "07c12d758307b4f394fa2fe5fa4d36bd"
server_key_version: "encryptionkey@0"
I20260927 22:57:45.697172 9047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260927 22:57:45.701187 9061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:45.702992 9047 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260927 22:57:45.703292 9047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "a51a175f120341d08400fcae9eeafedb"
format_stamp: "Formatted at 2026-09-27 22:57:45 on dist-test-slave-5pd3"
server_key: "c654b5d3504a3f8b56569f739270b979"
server_key_iv: "07c12d758307b4f394fa2fe5fa4d36bd"
server_key_version: "encryptionkey@0"
I20260927 22:57:45.703550 9047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:45.710108 9047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:45.710968 9047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:45.711292 9047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:45.732378 9047 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:40613
I20260927 22:57:45.732496 9112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:40613 every 8 connection(s)
I20260927 22:57:45.733933 9047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:45.739022 9113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:45.739636 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 9047
I20260927 22:57:45.739861 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:45.740583 4394 external_mini_cluster.cc:1428] Setting key ec7e9ff97a6015a17c7cb559b85a9353
I20260927 22:57:45.752185 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:45.743229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52404 (local address 127.4.74.190:40613)
0927 22:57:45.744775 (+ 1546us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:45.744796 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:45.744868 (+ 72us) server_negotiation.cc:407] Connection header received
0927 22:57:45.747299 (+ 2431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:45.747338 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:45.748149 (+ 811us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:45.748741 (+ 592us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:45.749954 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:45.749964 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:45.749982 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:45.750007 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:45.750089 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:45.750113 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:45.750372 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:45.750514 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 22:57:45.750858 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":525,"threads_started":1}
I20260927 22:57:45.756609 9113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb: Bootstrap starting.
I20260927 22:57:45.759927 9113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:45.761091 9113 log.cc:824] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:45.764775 9113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb: No bootstrap required, opened a new log
I20260927 22:57:45.772732 9113 raft_consensus.cc:348] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51a175f120341d08400fcae9eeafedb" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40613 } }
I20260927 22:57:45.773123 9113 raft_consensus.cc:374] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:45.773223 9113 raft_consensus.cc:729] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a51a175f120341d08400fcae9eeafedb, State: Initialized, Role: FOLLOWER
I20260927 22:57:45.773911 9113 consensus_queue.cc:260] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51a175f120341d08400fcae9eeafedb" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40613 } }
I20260927 22:57:45.774180 9113 raft_consensus.cc:388] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:45.774331 9113 raft_consensus.cc:482] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:45.774554 9113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:45.776829 9113 raft_consensus.cc:504] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51a175f120341d08400fcae9eeafedb" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40613 } }
I20260927 22:57:45.777355 9113 leader_election.cc:302] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a51a175f120341d08400fcae9eeafedb; no voters:
I20260927 22:57:45.778076 9113 leader_election.cc:288] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:45.778375 9118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:45.779439 9118 raft_consensus.cc:686] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [term 1 LEADER]: Becoming Leader. State: Replica: a51a175f120341d08400fcae9eeafedb, State: Running, Role: LEADER
I20260927 22:57:45.780135 9118 consensus_queue.cc:237] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51a175f120341d08400fcae9eeafedb" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40613 } }
I20260927 22:57:45.780705 9113 sys_catalog.cc:564] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:45.784351 9119 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a51a175f120341d08400fcae9eeafedb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51a175f120341d08400fcae9eeafedb" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40613 } } }
I20260927 22:57:45.784669 9120 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [sys.catalog]: SysCatalogTable state changed. Reason: New leader a51a175f120341d08400fcae9eeafedb. Latest consensus state: current_term: 1 leader_uuid: "a51a175f120341d08400fcae9eeafedb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51a175f120341d08400fcae9eeafedb" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40613 } } }
I20260927 22:57:45.784983 9119 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:45.785027 9120 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:45.787848 9126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:45.793910 9126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:45.805596 9126 catalog_manager.cc:1347] Generated new cluster ID: 099636792d5f4ea8a1ac96c685bd8a66
I20260927 22:57:45.805739 9126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:45.823474 9126 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:45.824249 9126 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:45.842373 9126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb: Generated new TSK 0
I20260927 22:57:45.843623 9126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 22:57:45.863101 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:40613
--builtin_ntp_servers=127.4.74.148:33567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 22:57:46.295051 9137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:46.295624 9137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:46.295768 9137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:46.304208 9137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:46.304354 9137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:46.304499 9137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:57:46.315721 9137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:33567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:46.317432 9137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:46.319769 9137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:46.336602 9142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:46.336687 9143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:46.336804 9145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:46.337824 9137 server_base.cc:1034] running on GCE node
I20260927 22:57:46.338686 9137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:46.340061 9137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:46.341353 9137 hybrid_clock.cc:648] HybridClock initialized: now 1790549866341265 us; error 73 us; skew 500 ppm
I20260927 22:57:46.341913 9137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:57:46.345207 9137 webserver.cc:460] Webserver started at http://127.4.74.129:35671/ using document root <none> and password file <none>
I20260927 22:57:46.346199 9137 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:46.346369 9137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:46.346772 9137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:46.350463 9137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "2315a69152b142a78bae29af0a86fc0c"
format_stamp: "Formatted at 2026-09-27 22:57:46 on dist-test-slave-5pd3"
server_key: "9c465c47d5800e23d3e782020c0b0490"
server_key_iv: "869ea8d10c9526e133a3c6b2ce5bd1d2"
server_key_version: "encryptionkey@0"
I20260927 22:57:46.351406 9137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "2315a69152b142a78bae29af0a86fc0c"
format_stamp: "Formatted at 2026-09-27 22:57:46 on dist-test-slave-5pd3"
server_key: "9c465c47d5800e23d3e782020c0b0490"
server_key_iv: "869ea8d10c9526e133a3c6b2ce5bd1d2"
server_key_version: "encryptionkey@0"
I20260927 22:57:46.358127 9137 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 22:57:46.363062 9151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:46.365222 9137 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260927 22:57:46.365492 9137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "2315a69152b142a78bae29af0a86fc0c"
format_stamp: "Formatted at 2026-09-27 22:57:46 on dist-test-slave-5pd3"
server_key: "9c465c47d5800e23d3e782020c0b0490"
server_key_iv: "869ea8d10c9526e133a3c6b2ce5bd1d2"
server_key_version: "encryptionkey@0"
I20260927 22:57:46.365820 9137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:46.373142 9137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:46.374249 9137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:46.374575 9137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:46.375813 9137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:46.377578 9137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:46.377712 9137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:46.377873 9137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:46.377964 9137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:46.418633 9137 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:37787
I20260927 22:57:46.418708 9263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:37787 every 8 connection(s)
I20260927 22:57:46.420354 9137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:57:46.421389 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 9137
I20260927 22:57:46.421607 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:57:46.422459 4394 external_mini_cluster.cc:1428] Setting key b66c766dffaa2409f9cda82826212eba
I20260927 22:57:46.426331 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:40613
--builtin_ntp_servers=127.4.74.148:33567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260927 22:57:46.430562 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:46.423971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:44139 (local address 127.4.74.190:40613)
0927 22:57:46.424504 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:46.424510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:46.426106 (+ 1596us) server_negotiation.cc:407] Connection header received
0927 22:57:46.428076 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:46.428082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:46.428147 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:46.428272 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:46.429781 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:46.429791 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:46.429794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:46.429809 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:46.429848 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:46.429860 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:46.430011 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:46.430203 (+ 192us) server_negotiation.cc:299] Negotiation successful
0927 22:57:46.430272 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":276,"threads_started":1}
I20260927 22:57:46.431670 9267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:46.424674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:40613 (local address 127.4.74.129:44139)
0927 22:57:46.425914 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:46.425968 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:46.427780 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:46.428527 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:46.428535 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:46.428636 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:46.429452 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:57:46.429454 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:46.429494 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:57:46.429600 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:46.429952 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:46.429962 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:46.429971 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 22:57:46.430289 (+ 318us) client_negotiation.cc:240] Negotiation successful
0927 22:57:46.430579 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":402,"threads_started":1}
I20260927 22:57:46.433871 9264 heartbeater.cc:347] Connected to a master server at 127.4.74.190:40613
I20260927 22:57:46.434247 9264 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:46.434916 9264 heartbeater.cc:511] Master 127.4.74.190:40613 requested a full tablet report, sending...
I20260927 22:57:46.437803 9078 ts_manager.cc:194] Registered new tserver with Master: 2315a69152b142a78bae29af0a86fc0c (127.4.74.129:37787)
W20260927 22:57:46.835484 9268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:46.835973 9268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:46.836081 9268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:46.845042 9268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:46.845208 9268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:46.845350 9268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:57:46.855818 9268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:33567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:46.857513 9268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:46.859820 9268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:46.877249 9274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:46.877156 9273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:46.877046 9276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:46.879205 9268 server_base.cc:1034] running on GCE node
I20260927 22:57:46.880244 9268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:46.881678 9268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:46.882964 9268 hybrid_clock.cc:648] HybridClock initialized: now 1790549866882886 us; error 65 us; skew 500 ppm
I20260927 22:57:46.883474 9268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:57:46.886608 9268 webserver.cc:460] Webserver started at http://127.4.74.130:39569/ using document root <none> and password file <none>
I20260927 22:57:46.887673 9268 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:46.887826 9268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:46.888269 9268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:46.892442 9268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "ac81a737c98548b49a20f98b4af0548f"
format_stamp: "Formatted at 2026-09-27 22:57:46 on dist-test-slave-5pd3"
server_key: "b98337858909497cb455600c6449c07e"
server_key_iv: "7720ae8a1a3ffef2947e3912ab949c8b"
server_key_version: "encryptionkey@0"
I20260927 22:57:46.893513 9268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "ac81a737c98548b49a20f98b4af0548f"
format_stamp: "Formatted at 2026-09-27 22:57:46 on dist-test-slave-5pd3"
server_key: "b98337858909497cb455600c6449c07e"
server_key_iv: "7720ae8a1a3ffef2947e3912ab949c8b"
server_key_version: "encryptionkey@0"
I20260927 22:57:46.900583 9268 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260927 22:57:46.905557 9282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:46.908017 9268 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260927 22:57:46.908288 9268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "ac81a737c98548b49a20f98b4af0548f"
format_stamp: "Formatted at 2026-09-27 22:57:46 on dist-test-slave-5pd3"
server_key: "b98337858909497cb455600c6449c07e"
server_key_iv: "7720ae8a1a3ffef2947e3912ab949c8b"
server_key_version: "encryptionkey@0"
I20260927 22:57:46.908640 9268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:46.917420 9268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:46.918591 9268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:46.918931 9268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:46.920305 9268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:46.922170 9268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:46.922312 9268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:46.922456 9268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:46.922529 9268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:46.965682 9268 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:33961
I20260927 22:57:46.965772 9394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:33961 every 8 connection(s)
I20260927 22:57:46.967592 9268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:57:46.974707 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 9268
I20260927 22:57:46.974900 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:57:46.975476 4394 external_mini_cluster.cc:1428] Setting key 93a91dafa32363569e7f4a264e63ea54
I20260927 22:57:46.978819 9397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:46.971159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:52011 (local address 127.4.74.190:40613)
0927 22:57:46.971735 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:46.971739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:46.973452 (+ 1713us) server_negotiation.cc:407] Connection header received
0927 22:57:46.976008 (+ 2556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:46.976015 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:46.976102 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:46.976328 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:46.977976 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:46.977982 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:46.977987 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:46.978019 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:46.978074 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:46.978087 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:46.978237 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:46.978441 (+ 204us) server_negotiation.cc:299] Negotiation successful
0927 22:57:46.978515 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":286,"threads_started":1}
I20260927 22:57:46.979219 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:40613
--builtin_ntp_servers=127.4.74.148:33567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260927 22:57:46.980059 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:46.971960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:40613 (local address 127.4.74.130:52011)
0927 22:57:46.973240 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:46.973297 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:46.975591 (+ 2294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:46.976639 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:46.976651 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:46.976809 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:46.977595 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:57:46.977599 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:46.977658 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:57:46.977786 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:46.978199 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:46.978212 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:46.978221 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 22:57:46.978564 (+ 343us) client_negotiation.cc:240] Negotiation successful
0927 22:57:46.978876 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":389,"threads_started":1}
I20260927 22:57:46.981986 9395 heartbeater.cc:347] Connected to a master server at 127.4.74.190:40613
I20260927 22:57:46.982362 9395 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:46.983076 9395 heartbeater.cc:511] Master 127.4.74.190:40613 requested a full tablet report, sending...
I20260927 22:57:46.985474 9078 ts_manager.cc:194] Registered new tserver with Master: ac81a737c98548b49a20f98b4af0548f (127.4.74.130:33961)
W20260927 22:57:47.361689 9399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:47.362059 9399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:47.362205 9399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:47.372520 9399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:47.372807 9399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:47.373062 9399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:57:47.384711 9399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:33567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:47.386234 9399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:47.388321 9399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:47.404672 9404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:47.405030 9405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:47.405613 9407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:47.406740 9399 server_base.cc:1034] running on GCE node
I20260927 22:57:47.407971 9399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:47.410034 9399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:47.412359 9399 hybrid_clock.cc:648] HybridClock initialized: now 1790549867412181 us; error 104 us; skew 500 ppm
I20260927 22:57:47.412925 9399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:57:47.416975 9399 webserver.cc:460] Webserver started at http://127.4.74.131:32855/ using document root <none> and password file <none>
I20260927 22:57:47.418347 9399 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:47.418567 9399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:47.419090 9399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:47.423337 9399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "6af08d30a4434780b905508b89a466e9"
format_stamp: "Formatted at 2026-09-27 22:57:47 on dist-test-slave-5pd3"
server_key: "a0c0d5941da8c625cc06e788013611b3"
server_key_iv: "4f9acd50b8d691a6d83e1a04f790202b"
server_key_version: "encryptionkey@0"
I20260927 22:57:47.424649 9399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "6af08d30a4434780b905508b89a466e9"
format_stamp: "Formatted at 2026-09-27 22:57:47 on dist-test-slave-5pd3"
server_key: "a0c0d5941da8c625cc06e788013611b3"
server_key_iv: "4f9acd50b8d691a6d83e1a04f790202b"
server_key_version: "encryptionkey@0"
I20260927 22:57:47.432618 9399 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260927 22:57:47.438920 9413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:47.441279 9399 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260927 22:57:47.441630 9399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "6af08d30a4434780b905508b89a466e9"
format_stamp: "Formatted at 2026-09-27 22:57:47 on dist-test-slave-5pd3"
server_key: "a0c0d5941da8c625cc06e788013611b3"
server_key_iv: "4f9acd50b8d691a6d83e1a04f790202b"
server_key_version: "encryptionkey@0"
I20260927 22:57:47.441962 9264 heartbeater.cc:503] Master 127.4.74.190:40613 was elected leader, sending a full tablet report...
I20260927 22:57:47.442065 9399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:47.450991 9399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:47.452874 9399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:47.453267 9399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:47.454954 9399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:47.457898 9399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:47.458158 9399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:47.458506 9399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:47.458586 9399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:47.500993 9399 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:44151
I20260927 22:57:47.501056 9525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:44151 every 8 connection(s)
I20260927 22:57:47.502614 9399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:57:47.509212 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 9399
I20260927 22:57:47.509416 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:57:47.509949 4394 external_mini_cluster.cc:1428] Setting key 8aeaffbe3782ec0fe62ccda22b1c3b99
I20260927 22:57:47.512957 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.506148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:41079 (local address 127.4.74.190:40613)
0927 22:57:47.506686 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:47.506691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:47.508038 (+ 1347us) server_negotiation.cc:407] Connection header received
0927 22:57:47.510184 (+ 2146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:47.510192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:47.510291 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:47.510485 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:47.512101 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:47.512107 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:47.512113 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:47.512138 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:47.512203 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:47.512228 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:47.512374 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:47.512559 (+ 185us) server_negotiation.cc:299] Negotiation successful
0927 22:57:47.512629 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":293,"threads_started":1}
I20260927 22:57:47.514158 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.506546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:40613 (local address 127.4.74.131:41079)
0927 22:57:47.507855 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:47.507900 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:47.509812 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:47.510733 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:47.510745 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:47.510875 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:47.511759 (+ 884us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:57:47.511765 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:47.511822 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:57:47.511913 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:47.512344 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:47.512361 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:47.512369 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 22:57:47.512654 (+ 285us) client_negotiation.cc:240] Negotiation successful
0927 22:57:47.512943 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":355,"threads_started":1}
I20260927 22:57:47.515858 9526 heartbeater.cc:347] Connected to a master server at 127.4.74.190:40613
I20260927 22:57:47.516310 9526 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:47.517133 9526 heartbeater.cc:511] Master 127.4.74.190:40613 requested a full tablet report, sending...
I20260927 22:57:47.519121 9078 ts_manager.cc:194] Registered new tserver with Master: 6af08d30a4434780b905508b89a466e9 (127.4.74.131:44151)
I20260927 22:57:47.527022 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 22:57:47.536369 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.533096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52412 (local address 127.4.74.190:40613)
0927 22:57:47.533378 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:47.533386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:47.533922 (+ 536us) server_negotiation.cc:407] Connection header received
0927 22:57:47.534195 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:47.534201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:47.534270 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:47.534393 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:47.535585 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:47.535592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:47.535598 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:47.535620 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:47.535666 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:47.535682 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:47.535839 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:47.535948 (+ 109us) server_negotiation.cc:299] Negotiation successful
0927 22:57:47.536026 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 22:57:47.541296 9078 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:57:47.548817 9078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:57:47.583828 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.566401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.4.74.131:44151)
0927 22:57:47.579513 (+ 13112us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:47.579524 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:47.579596 (+ 72us) server_negotiation.cc:407] Connection header received
0927 22:57:47.579709 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:47.579720 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:47.580016 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:47.580236 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:47.582158 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:47.582167 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:47.582183 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:47.582201 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:47.582267 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:47.582287 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:47.582476 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:47.583182 (+ 706us) server_negotiation.cc:299] Negotiation successful
0927 22:57:47.583328 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12846,"thread_start_us":12617,"threads_started":1}
I20260927 22:57:47.583863 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.565786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:33961 (local address 127.0.0.1:52256)
0927 22:57:47.576148 (+ 10362us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:47.576199 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:47.576412 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:47.580356 (+ 3944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:47.580365 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:47.580493 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:47.582636 (+ 2143us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:57:47.582640 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:47.582664 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:57:47.582710 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:47.583169 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:47.583174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:47.583179 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 22:57:47.583299 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 22:57:47.583341 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10029,"thread_start_us":13874,"threads_started":1}
I20260927 22:57:47.584271 9539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.565809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52256 (local address 127.4.74.130:33961)
0927 22:57:47.579538 (+ 13729us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:47.579576 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:47.579619 (+ 43us) server_negotiation.cc:407] Connection header received
0927 22:57:47.579729 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:47.579741 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:47.580087 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:47.581592 (+ 1505us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:47.582893 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:47.582902 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:47.582915 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:47.582944 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:47.583007 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:47.583038 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:47.583768 (+ 730us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:47.583845 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 22:57:47.583954 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13574,"thread_start_us":13528,"threads_started":1}
I20260927 22:57:47.584520 9543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.565964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:44151 (local address 127.0.0.1:48396)
0927 22:57:47.576938 (+ 10974us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:47.576971 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:47.577173 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:47.580367 (+ 3194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:47.580372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:47.580493 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:47.581822 (+ 1329us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:57:47.581825 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:47.581858 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:57:47.581944 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:47.582551 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:47.582565 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:47.582584 (+ 19us) client_negotiation.cc:769] Sending connection context
0927 22:57:47.582788 (+ 204us) client_negotiation.cc:240] Negotiation successful
0927 22:57:47.583215 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10725,"thread_start_us":9614,"threads_started":1}
I20260927 22:57:47.585431 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.567646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37787 (local address 127.0.0.1:55896)
0927 22:57:47.579989 (+ 12343us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:47.580013 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:47.580154 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:47.582259 (+ 2105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:47.582263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:47.582290 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:47.583783 (+ 1493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:57:47.583788 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:47.583819 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:57:47.583883 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:47.584587 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:47.584593 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:47.584599 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 22:57:47.584802 (+ 203us) client_negotiation.cc:240] Negotiation successful
0927 22:57:47.584870 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12218,"thread_start_us":8877,"threads_started":1}
I20260927 22:57:47.586374 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.567796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.4.74.129:37787)
0927 22:57:47.579089 (+ 11293us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:47.579102 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:47.580155 (+ 1053us) server_negotiation.cc:407] Connection header received
0927 22:57:47.580368 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:47.580377 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:47.580580 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:47.580725 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:47.584156 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:47.584167 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:47.584184 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:47.584209 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:47.584288 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:47.584312 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:47.584522 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:47.585796 (+ 1274us) server_negotiation.cc:299] Negotiation successful
0927 22:57:47.585928 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11008,"thread_start_us":10721,"threads_started":1}
I20260927 22:57:47.588944 9461 tablet_service.cc:1467] Processing CreateTablet for tablet 622043ac883847b29e9b05d0fc92b621 (DEFAULT_TABLE table=test-table [id=ca52e350ab744a488d16807cf8056227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:47.588944 9330 tablet_service.cc:1467] Processing CreateTablet for tablet 622043ac883847b29e9b05d0fc92b621 (DEFAULT_TABLE table=test-table [id=ca52e350ab744a488d16807cf8056227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:47.590590 9461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 622043ac883847b29e9b05d0fc92b621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:47.590585 9330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 622043ac883847b29e9b05d0fc92b621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:47.591362 9199 tablet_service.cc:1467] Processing CreateTablet for tablet 622043ac883847b29e9b05d0fc92b621 (DEFAULT_TABLE table=test-table [id=ca52e350ab744a488d16807cf8056227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:57:47.593294 9199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 622043ac883847b29e9b05d0fc92b621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:47.609261 9545 tablet_bootstrap.cc:492] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: Bootstrap starting.
I20260927 22:57:47.609869 9547 tablet_bootstrap.cc:492] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c: Bootstrap starting.
I20260927 22:57:47.609896 9546 tablet_bootstrap.cc:492] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: Bootstrap starting.
I20260927 22:57:47.614023 9546 tablet_bootstrap.cc:654] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:47.614020 9545 tablet_bootstrap.cc:654] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:47.614182 9547 tablet_bootstrap.cc:654] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:47.615477 9546 log.cc:824] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:47.615489 9545 log.cc:824] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:47.615882 9547 log.cc:824] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:47.625322 9546 tablet_bootstrap.cc:492] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: No bootstrap required, opened a new log
I20260927 22:57:47.625686 9546 ts_tablet_manager.cc:1397] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260927 22:57:47.626009 9547 tablet_bootstrap.cc:492] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c: No bootstrap required, opened a new log
I20260927 22:57:47.626010 9545 tablet_bootstrap.cc:492] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: No bootstrap required, opened a new log
I20260927 22:57:47.626399 9547 ts_tablet_manager.cc:1397] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260927 22:57:47.626399 9545 ts_tablet_manager.cc:1397] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.000s
I20260927 22:57:47.633770 9546 raft_consensus.cc:348] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac81a737c98548b49a20f98b4af0548f" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 33961 } } peers { permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 } } peers { permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 } }
I20260927 22:57:47.634218 9546 raft_consensus.cc:374] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:47.634327 9546 raft_consensus.cc:729] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac81a737c98548b49a20f98b4af0548f, State: Initialized, Role: FOLLOWER
I20260927 22:57:47.635030 9546 consensus_queue.cc:260] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac81a737c98548b49a20f98b4af0548f" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 33961 } } peers { permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 } } peers { permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 } }
I20260927 22:57:47.636219 9395 heartbeater.cc:503] Master 127.4.74.190:40613 was elected leader, sending a full tablet report...
I20260927 22:57:47.638267 9546 ts_tablet_manager.cc:1428] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: Time spent starting tablet: real 0.012s user 0.007s sys 0.006s
I20260927 22:57:47.639605 9547 raft_consensus.cc:348] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac81a737c98548b49a20f98b4af0548f" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 33961 } } peers { permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 } } peers { permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 } }
I20260927 22:57:47.639621 9545 raft_consensus.cc:348] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac81a737c98548b49a20f98b4af0548f" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 33961 } } peers { permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 } } peers { permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 } }
I20260927 22:57:47.640259 9545 raft_consensus.cc:374] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:47.640321 9547 raft_consensus.cc:374] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:47.640468 9547 raft_consensus.cc:729] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2315a69152b142a78bae29af0a86fc0c, State: Initialized, Role: FOLLOWER
I20260927 22:57:47.640467 9545 raft_consensus.cc:729] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af08d30a4434780b905508b89a466e9, State: Initialized, Role: FOLLOWER
I20260927 22:57:47.641331 9545 consensus_queue.cc:260] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac81a737c98548b49a20f98b4af0548f" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 33961 } } peers { permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 } } peers { permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 } }
I20260927 22:57:47.641331 9547 consensus_queue.cc:260] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac81a737c98548b49a20f98b4af0548f" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 33961 } } peers { permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 } } peers { permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 } }
I20260927 22:57:47.643330 9526 heartbeater.cc:503] Master 127.4.74.190:40613 was elected leader, sending a full tablet report...
I20260927 22:57:47.643929 9547 ts_tablet_manager.cc:1428] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c: Time spent starting tablet: real 0.017s user 0.015s sys 0.005s
I20260927 22:57:47.643947 9545 ts_tablet_manager.cc:1428] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: Time spent starting tablet: real 0.017s user 0.012s sys 0.008s
W20260927 22:57:47.673267 9265 tablet.cc:2369] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 22:57:47.720206 9396 tablet.cc:2369] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:57:47.724766 9551 raft_consensus.cc:482] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:57:47.725221 9551 raft_consensus.cc:504] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac81a737c98548b49a20f98b4af0548f" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 33961 } } peers { permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 } } peers { permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 } }
I20260927 22:57:47.727742 9551 leader_election.cc:288] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6af08d30a4434780b905508b89a466e9 (127.4.74.131:44151), 2315a69152b142a78bae29af0a86fc0c (127.4.74.129:37787)
I20260927 22:57:47.732394 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.728077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37787 (local address 127.4.74.130:39511)
0927 22:57:47.729867 (+ 1790us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:47.729890 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:47.730062 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:47.730434 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:47.730439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:47.730466 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:47.731184 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:57:47.731188 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:47.731208 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:57:47.731260 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:47.731722 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:47.731727 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:47.731732 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 22:57:47.731909 (+ 177us) client_negotiation.cc:240] Negotiation successful
0927 22:57:47.731979 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1614,"mutex_wait_us":982,"thread_start_us":365,"threads_started":1}
I20260927 22:57:47.732388 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.728352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:39511 (local address 127.4.74.129:37787)
0927 22:57:47.728666 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:47.728673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:47.729996 (+ 1323us) server_negotiation.cc:407] Connection header received
0927 22:57:47.730229 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:47.730238 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:47.730291 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:47.730403 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:47.731467 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:47.731476 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:47.731481 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:47.731505 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:47.731600 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:47.731618 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:47.731739 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:47.731870 (+ 131us) server_negotiation.cc:299] Negotiation successful
0927 22:57:47.731955 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260927 22:57:47.732422 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.729029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:52541 (local address 127.4.74.131:44151)
0927 22:57:47.729238 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:47.729246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:47.729490 (+ 244us) server_negotiation.cc:407] Connection header received
0927 22:57:47.729757 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:47.729762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:47.729831 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:47.729947 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:47.731207 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:47.731219 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:47.731222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:47.731239 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:47.731277 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:47.731290 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:47.731456 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:47.731973 (+ 517us) server_negotiation.cc:299] Negotiation successful
0927 22:57:47.732153 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260927 22:57:47.733280 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.728077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:44151 (local address 127.4.74.130:52541)
0927 22:57:47.729313 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:47.729346 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:47.729573 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:47.729978 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:47.729983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:47.730011 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:47.730918 (+ 907us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:57:47.730923 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:47.730946 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:57:47.731006 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:47.731787 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:47.731790 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:47.731816 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 22:57:47.732696 (+ 880us) client_negotiation.cc:240] Negotiation successful
0927 22:57:47.732788 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"thread_start_us":641,"threads_started":1}
I20260927 22:57:47.734473 9481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622043ac883847b29e9b05d0fc92b621" candidate_uuid: "ac81a737c98548b49a20f98b4af0548f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6af08d30a4434780b905508b89a466e9" is_pre_election: true
I20260927 22:57:47.734433 9219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622043ac883847b29e9b05d0fc92b621" candidate_uuid: "ac81a737c98548b49a20f98b4af0548f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2315a69152b142a78bae29af0a86fc0c" is_pre_election: true
I20260927 22:57:47.735158 9219 raft_consensus.cc:2454] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac81a737c98548b49a20f98b4af0548f in term 0.
I20260927 22:57:47.735158 9481 raft_consensus.cc:2454] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac81a737c98548b49a20f98b4af0548f in term 0.
I20260927 22:57:47.736109 9284 leader_election.cc:302] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6af08d30a4434780b905508b89a466e9, ac81a737c98548b49a20f98b4af0548f; no voters:
I20260927 22:57:47.736642 9551 raft_consensus.cc:2781] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:57:47.736799 9551 raft_consensus.cc:482] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:57:47.736919 9551 raft_consensus.cc:3037] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:47.739491 9551 raft_consensus.cc:504] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac81a737c98548b49a20f98b4af0548f" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 33961 } } peers { permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 } } peers { permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 } }
I20260927 22:57:47.740326 9551 leader_election.cc:288] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [CANDIDATE]: Term 1 election: Requested vote from peers 6af08d30a4434780b905508b89a466e9 (127.4.74.131:44151), 2315a69152b142a78bae29af0a86fc0c (127.4.74.129:37787)
I20260927 22:57:47.740859 9481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622043ac883847b29e9b05d0fc92b621" candidate_uuid: "ac81a737c98548b49a20f98b4af0548f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6af08d30a4434780b905508b89a466e9"
I20260927 22:57:47.741142 9481 raft_consensus.cc:3037] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:47.741231 9219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622043ac883847b29e9b05d0fc92b621" candidate_uuid: "ac81a737c98548b49a20f98b4af0548f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2315a69152b142a78bae29af0a86fc0c"
I20260927 22:57:47.741510 9219 raft_consensus.cc:3037] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:47.745311 9219 raft_consensus.cc:2454] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac81a737c98548b49a20f98b4af0548f in term 1.
I20260927 22:57:47.745302 9481 raft_consensus.cc:2454] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac81a737c98548b49a20f98b4af0548f in term 1.
I20260927 22:57:47.745990 9284 leader_election.cc:302] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6af08d30a4434780b905508b89a466e9, ac81a737c98548b49a20f98b4af0548f; no voters:
I20260927 22:57:47.746431 9551 raft_consensus.cc:2781] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:47.747015 9551 raft_consensus.cc:686] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 1 LEADER]: Becoming Leader. State: Replica: ac81a737c98548b49a20f98b4af0548f, State: Running, Role: LEADER
I20260927 22:57:47.747753 9551 consensus_queue.cc:237] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac81a737c98548b49a20f98b4af0548f" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 33961 } } peers { permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 } } peers { permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 } }
I20260927 22:57:47.753736 9077 catalog_manager.cc:5495] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f reported cstate change: term changed from 0 to 1, leader changed from <none> to ac81a737c98548b49a20f98b4af0548f (127.4.74.130). New cstate: current_term: 1 leader_uuid: "ac81a737c98548b49a20f98b4af0548f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac81a737c98548b49a20f98b4af0548f" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 } health_report { overall_health: UNKNOWN } } }
W20260927 22:57:47.754748 9527 tablet.cc:2369] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:57:47.787990 9539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.785209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52272 (local address 127.4.74.130:33961)
0927 22:57:47.785514 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:47.785523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:47.785667 (+ 144us) server_negotiation.cc:407] Connection header received
0927 22:57:47.785882 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:47.785887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:47.785993 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:47.786132 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:47.787227 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:47.787232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:47.787236 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:47.787252 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:47.787318 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:47.787331 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:47.787438 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:47.787523 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 22:57:47.787643 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260927 22:57:47.798249 9219 raft_consensus.cc:1261] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c [term 1 FOLLOWER]: Refusing update from remote peer ac81a737c98548b49a20f98b4af0548f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:47.798300 9481 raft_consensus.cc:1261] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9 [term 1 FOLLOWER]: Refusing update from remote peer ac81a737c98548b49a20f98b4af0548f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:57:47.799407 9556 consensus_queue.cc:1035] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6af08d30a4434780b905508b89a466e9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:47.799942 9551 consensus_queue.cc:1035] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2315a69152b142a78bae29af0a86fc0c" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 37787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:57:47.811172 9559 mvcc.cc:205] Tried to move back new op lower bound from 7334092258487889920 to 7334092258300362752. Current Snapshot: MvccSnapshot[applied={T|T < 7334092258487889920}]
I20260927 22:57:47.820874 9562 mvcc.cc:205] Tried to move back new op lower bound from 7334092258487889920 to 7334092258300362752. Current Snapshot: MvccSnapshot[applied={T|T < 7334092258487889920}]
I20260927 22:57:47.831630 9078 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:47.831929 9078 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:57:47.836732 9078 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a51a175f120341d08400fcae9eeafedb: Sending DeleteTablet for 3 replicas of tablet 622043ac883847b29e9b05d0fc92b621
I20260927 22:57:47.838409 9330 tablet_service.cc:1514] Processing DeleteTablet for tablet 622043ac883847b29e9b05d0fc92b621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:47 UTC) from {username='slave'} at 127.0.0.1:52256
I20260927 22:57:47.838403 9461 tablet_service.cc:1514] Processing DeleteTablet for tablet 622043ac883847b29e9b05d0fc92b621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:47 UTC) from {username='slave'} at 127.0.0.1:48396
I20260927 22:57:47.838610 9199 tablet_service.cc:1514] Processing DeleteTablet for tablet 622043ac883847b29e9b05d0fc92b621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:57:47 UTC) from {username='slave'} at 127.0.0.1:55896
I20260927 22:57:47.842523 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 9137
I20260927 22:57:47.849092 9570 tablet_replica.cc:330] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c: stopping tablet replica
I20260927 22:57:47.849807 9570 raft_consensus.cc:2229] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:47.850509 9570 raft_consensus.cc:2258] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:47.851707 9568 tablet_replica.cc:330] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: stopping tablet replica
I20260927 22:57:47.852248 9568 raft_consensus.cc:2229] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:57:47.853124 9568 raft_consensus.cc:2258] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:47.854072 9569 tablet_replica.cc:330] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: stopping tablet replica
I20260927 22:57:47.854068 9570 ts_tablet_manager.cc:1905] T 622043ac883847b29e9b05d0fc92b621 P 2315a69152b142a78bae29af0a86fc0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:47.854576 9569 raft_consensus.cc:2229] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:57:47.855580 9569 raft_consensus.cc:2258] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:57:47.857780 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:47.845733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55902 (local address 127.4.74.129:37787)
0927 22:57:47.846008 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:47.846018 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:47.848580 (+ 2562us) server_negotiation.cc:407] Connection header received
0927 22:57:47.853900 (+ 5320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:47.853909 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:47.854039 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:47.854166 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:47.856789 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:47.856796 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:47.856801 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:47.856829 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:47.856880 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:47.856900 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:47.857050 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:47.857342 (+ 292us) server_negotiation.cc:299] Negotiation successful
0927 22:57:47.857427 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260927 22:57:47.858353 9569 ts_tablet_manager.cc:1905] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:47.859154 9259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:47.860414 9568 ts_tablet_manager.cc:1905] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:57:47.866561 9569 ts_tablet_manager.cc:1918] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:57:47.866804 9569 log.cc:1196] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/622043ac883847b29e9b05d0fc92b621
I20260927 22:57:47.867408 9569 ts_tablet_manager.cc:1939] T 622043ac883847b29e9b05d0fc92b621 P 6af08d30a4434780b905508b89a466e9: Deleting consensus metadata
I20260927 22:57:47.869444 9568 ts_tablet_manager.cc:1918] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:57:47.869673 9568 log.cc:1196] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/622043ac883847b29e9b05d0fc92b621
I20260927 22:57:47.870076 9063 catalog_manager.cc:4841] TS 6af08d30a4434780b905508b89a466e9 (127.4.74.131:44151): tablet 622043ac883847b29e9b05d0fc92b621 (table test-table [id=ca52e350ab744a488d16807cf8056227]) successfully deleted
I20260927 22:57:47.871120 9568 ts_tablet_manager.cc:1939] T 622043ac883847b29e9b05d0fc92b621 P ac81a737c98548b49a20f98b4af0548f: Deleting consensus metadata
I20260927 22:57:47.874054 9065 catalog_manager.cc:4841] TS ac81a737c98548b49a20f98b4af0548f (127.4.74.130:33961): tablet 622043ac883847b29e9b05d0fc92b621 (table test-table [id=ca52e350ab744a488d16807cf8056227]) successfully deleted
I20260927 22:57:47.974714 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 9137
W20260927 22:57:48.002049 9064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.74.129:37787 (error 108)
I20260927 22:57:48.002892 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 9268
I20260927 22:57:48.004364 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:48.003831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37787 (local address 127.0.0.1:55916)
0927 22:57:48.004022 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:48.004176 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:37787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260927 22:57:48.004700 9064 catalog_manager.cc:4584] TS 2315a69152b142a78bae29af0a86fc0c (127.4.74.129:37787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 622043ac883847b29e9b05d0fc92b621: Network error: Client connection negotiation failed: client connection to 127.4.74.129:37787: connect: Connection refused (error 111)
I20260927 22:57:48.007225 9539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:48.004770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52286 (local address 127.4.74.130:33961)
0927 22:57:48.005061 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:48.005068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:48.005096 (+ 28us) server_negotiation.cc:407] Connection header received
0927 22:57:48.005219 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:48.005229 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:48.005331 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:48.005484 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:48.006448 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:48.006455 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:48.006460 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:48.006482 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:48.006532 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:48.006548 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:48.006699 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:48.006811 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 22:57:48.006881 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260927 22:57:48.008261 9390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:48.058853 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:48.058102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37787 (local address 127.0.0.1:55924)
0927 22:57:48.058482 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:48.058623 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:37787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260927 22:57:48.060177 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:48.059803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37787 (local address 127.0.0.1:55936)
0927 22:57:48.059958 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:48.060042 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:37787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260927 22:57:48.121351 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:48.120690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37787 (local address 127.0.0.1:55952)
0927 22:57:48.120993 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:48.121127 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:37787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260927 22:57:48.122583 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:48.122211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37787 (local address 127.0.0.1:55964)
0927 22:57:48.122397 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:48.122465 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:37787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 22:57:48.129065 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 9268
I20260927 22:57:48.148216 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 9399
I20260927 22:57:48.157189 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:48.149147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48406 (local address 127.4.74.131:44151)
0927 22:57:48.149367 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:48.149374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:48.149494 (+ 120us) server_negotiation.cc:407] Connection header received
0927 22:57:48.149720 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:48.149726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:48.149827 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:48.149936 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:48.156484 (+ 6548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:48.156489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:48.156492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:48.156513 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:48.156551 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:48.156565 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:48.156729 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:48.156789 (+ 60us) server_negotiation.cc:299] Negotiation successful
0927 22:57:48.156872 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 22:57:48.158008 9521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:48.203514 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:48.202813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37787 (local address 127.0.0.1:55978)
0927 22:57:48.203171 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:48.203292 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:37787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200}
I20260927 22:57:48.205229 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:48.204398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:37787 (local address 127.0.0.1:55988)
0927 22:57:48.204767 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:48.204927 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:37787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260927 22:57:48.254706 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 9399
I20260927 22:57:48.273885 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 9047
I20260927 22:57:48.275482 9108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:48.383420 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 9047
2026-09-27T22:57:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3151 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260927 22:57:48.429515 4394 mini_oidc.cc:152] Starting JWKS server
I20260927 22:57:48.432410 4394 webserver.cc:460] Webserver started at https://127.0.0.1:33191/ using document root <none> and password file <none>
I20260927 22:57:48.432760 4394 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 22:57:48.433511 4394 webserver.cc:460] Webserver started at http://127.0.0.1:35089/ using document root <none> and password file <none>
2026-09-27T22:57:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:48Z Disabled control of system clock
I20260927 22:57:48.462813 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:45041
--webserver_interface=127.4.74.190
--webserver_port=0
--builtin_ntp_servers=127.4.74.148:41507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:45041
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33191/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/testchainca.cert with env {}
W20260927 22:57:48.790598 9582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:48.790985 9582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:48.791157 9582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:48.798810 9582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:48.798974 9582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:48.799124 9582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:48.799243 9582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:48.799332 9582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 22:57:48.799424 9582 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 22:57:48.799503 9582 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33191/jwks.json
I20260927 22:57:48.810710 9582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:45041
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:45041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33191/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:48.812243 9582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:48.813583 9582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:48.826845 9587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:48.826979 9588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:48.827458 9590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:48.828351 9582 server_base.cc:1034] running on GCE node
I20260927 22:57:48.829491 9582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:48.831782 9582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:48.833078 9582 hybrid_clock.cc:648] HybridClock initialized: now 1790549868833045 us; error 69 us; skew 500 ppm
I20260927 22:57:48.833564 9582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:57:48.836723 9582 webserver.cc:460] Webserver started at http://127.4.74.190:33985/ using document root <none> and password file <none>
I20260927 22:57:48.837641 9582 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:48.837793 9582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:48.838160 9582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:48.841792 9582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "c767dbe3a7d042f896ce297d51c7a727"
format_stamp: "Formatted at 2026-09-27 22:57:48 on dist-test-slave-5pd3"
server_key: "e7563e40a972af9b24b8bf579ab1c5b4"
server_key_iv: "df826bb49aafe2e7183a56ab5809c5f3"
server_key_version: "encryptionkey@0"
I20260927 22:57:48.842736 9582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "c767dbe3a7d042f896ce297d51c7a727"
format_stamp: "Formatted at 2026-09-27 22:57:48 on dist-test-slave-5pd3"
server_key: "e7563e40a972af9b24b8bf579ab1c5b4"
server_key_iv: "df826bb49aafe2e7183a56ab5809c5f3"
server_key_version: "encryptionkey@0"
I20260927 22:57:48.849499 9582 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260927 22:57:48.854068 9596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:48.856436 9582 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 22:57:48.856750 9582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "c767dbe3a7d042f896ce297d51c7a727"
format_stamp: "Formatted at 2026-09-27 22:57:48 on dist-test-slave-5pd3"
server_key: "e7563e40a972af9b24b8bf579ab1c5b4"
server_key_iv: "df826bb49aafe2e7183a56ab5809c5f3"
server_key_version: "encryptionkey@0"
I20260927 22:57:48.857060 9582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:48.883162 9582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:48.884423 9582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:48.884814 9582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:48.909423 9582 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:45041
I20260927 22:57:48.909464 9647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:45041 every 8 connection(s)
I20260927 22:57:48.911453 9582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:48.916015 9648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:48.917532 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 9582
I20260927 22:57:48.917824 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:48.918536 4394 external_mini_cluster.cc:1428] Setting key cd7c146a835885b10e92957db09bef9e
I20260927 22:57:48.930199 9648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727: Bootstrap starting.
I20260927 22:57:48.935246 9648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:48.937057 9648 log.cc:824] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:48.937306 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:48.921316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52414 (local address 127.4.74.190:45041)
0927 22:57:48.922884 (+ 1568us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:48.922908 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:48.922969 (+ 61us) server_negotiation.cc:407] Connection header received
0927 22:57:48.925271 (+ 2302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:48.925306 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:48.926062 (+ 756us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:48.926565 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:48.927508 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:48.929255 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:48.931697 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:48.932584 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:48.932775 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:48.934437 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:48.934446 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:48.934471 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:48.935134 (+ 663us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:48.935246 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:48.935268 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:48.935401 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:48.935713 (+ 312us) server_negotiation.cc:299] Negotiation successful
0927 22:57:48.936173 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":983,"thread_start_us":639,"threads_started":1}
I20260927 22:57:48.941279 9648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727: No bootstrap required, opened a new log
I20260927 22:57:48.941880 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 22:57:48.951232 9648 raft_consensus.cc:348] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767dbe3a7d042f896ce297d51c7a727" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45041 } }
I20260927 22:57:48.951865 9648 raft_consensus.cc:374] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:48.951963 9648 raft_consensus.cc:729] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c767dbe3a7d042f896ce297d51c7a727, State: Initialized, Role: FOLLOWER
I20260927 22:57:48.952822 9648 consensus_queue.cc:260] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767dbe3a7d042f896ce297d51c7a727" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45041 } }
I20260927 22:57:48.953135 9648 raft_consensus.cc:388] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:48.953300 9648 raft_consensus.cc:482] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:48.953526 9648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:48.956146 9648 raft_consensus.cc:504] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767dbe3a7d042f896ce297d51c7a727" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45041 } }
I20260927 22:57:48.956749 9648 leader_election.cc:302] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c767dbe3a7d042f896ce297d51c7a727; no voters:
I20260927 22:57:48.957438 9648 leader_election.cc:288] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:48.957752 9653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:48.958600 9653 raft_consensus.cc:686] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [term 1 LEADER]: Becoming Leader. State: Replica: c767dbe3a7d042f896ce297d51c7a727, State: Running, Role: LEADER
I20260927 22:57:48.959237 9653 consensus_queue.cc:237] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767dbe3a7d042f896ce297d51c7a727" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45041 } }
I20260927 22:57:48.960251 9648 sys_catalog.cc:564] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:48.962443 9655 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c767dbe3a7d042f896ce297d51c7a727. Latest consensus state: current_term: 1 leader_uuid: "c767dbe3a7d042f896ce297d51c7a727" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767dbe3a7d042f896ce297d51c7a727" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45041 } } }
I20260927 22:57:48.962896 9655 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:48.962745 9654 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c767dbe3a7d042f896ce297d51c7a727" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767dbe3a7d042f896ce297d51c7a727" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45041 } } }
I20260927 22:57:48.963282 9654 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:48.966642 9658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:48.974606 9658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:48.985841 9658 catalog_manager.cc:1347] Generated new cluster ID: 4e1c756bc954436d932cbc787744de3d
I20260927 22:57:48.985993 9658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:49.011674 9658 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:49.014194 9658 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:49.030077 9658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c767dbe3a7d042f896ce297d51c7a727: Generated new TSK 0
I20260927 22:57:49.031319 9658 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 22:57:49.090178 9678 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260927 22:57:49.090437 9678 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 22:57:49.091845 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:49.058028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.4.74.190:45041)
0927 22:57:49.058339 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:49.058360 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:49.058920 (+ 560us) server_negotiation.cc:407] Connection header received
0927 22:57:49.059390 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:49.059396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:49.059500 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:49.059776 (+ 276us) server_negotiation.cc:226] Negotiated authn=JWT
0927 22:57:49.060604 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:49.061715 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:49.064015 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:49.064518 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:49.064637 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:49.065376 (+ 739us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 22:57:49.089464 (+ 24088us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33191/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 22:57:49.091359 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52418: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33191/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":94}
W20260927 22:57:49.092092 9651 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52418: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33191/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260927 22:57:49.092635 9677 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:57:49.057917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:45041 (local address 127.0.0.1:52418)
0927 22:57:49.058747 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:49.058771 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:49.059001 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:49.059835 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:49.059842 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:49.059857 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0927 22:57:49.060327 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:49.060340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:49.061894 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:49.061901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:49.063818 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:49.063833 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:49.064719 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:49.064725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:49.064909 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:49.065002 (+ 93us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 22:57:49.089890 (+ 24888us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33191/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 22:57:49.090316 (+ 426us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.74.190:45041: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33191/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":425,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.74.190:45041: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33191/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260927 22:57:49.095086 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 9582
I20260927 22:57:49.097025 9643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:49.226428 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 9582
2026-09-27T22:57:49Z chronyd exiting
I20260927 22:57:49.278702 4394 test_util.cc:175] -----------------------------------------------
I20260927 22:57:49.278877 4394 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790549821755302-4394-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (854 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260927 22:57:49.283146 4394 mini_oidc.cc:152] Starting JWKS server
I20260927 22:57:49.285451 4394 webserver.cc:460] Webserver started at https://127.0.0.1:42793/ using document root <none> and password file <none>
I20260927 22:57:49.285765 4394 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 22:57:49.286564 4394 webserver.cc:460] Webserver started at http://127.0.0.1:44511/ using document root <none> and password file <none>
2026-09-27T22:57:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:49Z Disabled control of system clock
I20260927 22:57:49.320779 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:45827
--webserver_interface=127.4.74.190
--webserver_port=0
--builtin_ntp_servers=127.4.74.148:42219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:45827
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42793/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/testchainca.cert with env {}
W20260927 22:57:49.651578 9685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:49.651953 9685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:49.652127 9685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:49.659518 9685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:49.659719 9685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:49.659819 9685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:49.659926 9685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:49.660012 9685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 22:57:49.660111 9685 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 22:57:49.660192 9685 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42793/jwks.json
I20260927 22:57:49.670370 9685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:42219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:45827
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:45827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42793/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:49.671828 9685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:49.673180 9685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:49.685889 9691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:49.686061 9690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:49.685894 9693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:49.687156 9685 server_base.cc:1034] running on GCE node
I20260927 22:57:49.688081 9685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:49.689507 9685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:49.690732 9685 hybrid_clock.cc:648] HybridClock initialized: now 1790549869690680 us; error 50 us; skew 500 ppm
I20260927 22:57:49.691148 9685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:57:49.693652 9685 webserver.cc:460] Webserver started at http://127.4.74.190:36587/ using document root <none> and password file <none>
I20260927 22:57:49.694383 9685 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:49.694478 9685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:49.694842 9685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:49.697762 9685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3"
format_stamp: "Formatted at 2026-09-27 22:57:49 on dist-test-slave-5pd3"
server_key: "b24afda82689306ed8cba0033bff0f55"
server_key_iv: "b719266d134a53e5e163f4ef128bd8f1"
server_key_version: "encryptionkey@0"
I20260927 22:57:49.698786 9685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3"
format_stamp: "Formatted at 2026-09-27 22:57:49 on dist-test-slave-5pd3"
server_key: "b24afda82689306ed8cba0033bff0f55"
server_key_iv: "b719266d134a53e5e163f4ef128bd8f1"
server_key_version: "encryptionkey@0"
I20260927 22:57:49.704689 9685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260927 22:57:49.708564 9699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:49.710408 9685 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:57:49.710659 9685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3"
format_stamp: "Formatted at 2026-09-27 22:57:49 on dist-test-slave-5pd3"
server_key: "b24afda82689306ed8cba0033bff0f55"
server_key_iv: "b719266d134a53e5e163f4ef128bd8f1"
server_key_version: "encryptionkey@0"
I20260927 22:57:49.710922 9685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:49.736631 9685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:49.737517 9685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:49.737843 9685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:49.759328 9685 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:45827
I20260927 22:57:49.759372 9750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:45827 every 8 connection(s)
I20260927 22:57:49.760879 9685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:49.766430 9751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:49.766389 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 9685
I20260927 22:57:49.766623 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:49.767508 4394 external_mini_cluster.cc:1428] Setting key 9860d7820ca31a44f2e18a2911d5257f
I20260927 22:57:49.783861 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:49.770009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53384 (local address 127.4.74.190:45827)
0927 22:57:49.771705 (+ 1696us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:49.771720 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:49.771781 (+ 61us) server_negotiation.cc:407] Connection header received
0927 22:57:49.773642 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:49.773669 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:49.774281 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:49.774821 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:49.775513 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:49.777064 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:49.778496 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:49.779303 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:49.779589 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:49.781170 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:49.781182 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:49.781206 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:49.781831 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:49.781925 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:49.781954 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:49.782112 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:49.782364 (+ 252us) server_negotiation.cc:299] Negotiation successful
0927 22:57:49.782719 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":968,"thread_start_us":561,"threads_started":1}
I20260927 22:57:49.783916 9751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3: Bootstrap starting.
I20260927 22:57:49.788269 9751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:49.788398 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 22:57:49.789580 9751 log.cc:824] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:49.793392 9751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3: No bootstrap required, opened a new log
I20260927 22:57:49.802989 9751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45827 } }
I20260927 22:57:49.803529 9751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:49.803746 9751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c016bc1ee2a4816a8b94f2df6fa0fa3, State: Initialized, Role: FOLLOWER
I20260927 22:57:49.804477 9751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45827 } }
I20260927 22:57:49.804720 9751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:49.804927 9751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:49.805114 9751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:49.807787 9751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45827 } }
I20260927 22:57:49.808521 9751 leader_election.cc:302] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c016bc1ee2a4816a8b94f2df6fa0fa3; no voters:
I20260927 22:57:49.809571 9751 leader_election.cc:288] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:49.809666 9756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:49.811170 9756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [term 1 LEADER]: Becoming Leader. State: Replica: 2c016bc1ee2a4816a8b94f2df6fa0fa3, State: Running, Role: LEADER
I20260927 22:57:49.811916 9756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45827 } }
I20260927 22:57:49.812641 9751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:49.816301 9758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c016bc1ee2a4816a8b94f2df6fa0fa3. Latest consensus state: current_term: 1 leader_uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45827 } } }
I20260927 22:57:49.817020 9758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:49.817996 9763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:49.819128 9757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c016bc1ee2a4816a8b94f2df6fa0fa3" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45827 } } }
I20260927 22:57:49.819680 9757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:49.823751 9763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:49.836433 9763 catalog_manager.cc:1347] Generated new cluster ID: 66d1cf24757840dd9c19e3bcfd0f0f2f
I20260927 22:57:49.836680 9763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:49.862676 9763 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:49.864235 9763 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:49.875747 9763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c016bc1ee2a4816a8b94f2df6fa0fa3: Generated new TSK 0
I20260927 22:57:49.877383 9763 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 22:57:49.934433 9781 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260927 22:57:49.934630 9781 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 22:57:49.935711 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:49.903855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.4.74.190:45827)
0927 22:57:49.904167 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:49.904176 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:49.904732 (+ 556us) server_negotiation.cc:407] Connection header received
0927 22:57:49.905110 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:49.905116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:49.905214 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:49.905385 (+ 171us) server_negotiation.cc:226] Negotiated authn=JWT
0927 22:57:49.906093 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:49.906916 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:49.909504 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:49.910255 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:49.910399 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:49.911244 (+ 845us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 22:57:49.934987 (+ 23743us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42793/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 22:57:49.935364 (+ 377us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53392: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42793/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":101}
W20260927 22:57:49.935878 9754 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53392: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42793/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260927 22:57:49.936355 9780 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:57:49.903701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:45827 (local address 127.0.0.1:53392)
0927 22:57:49.904559 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:49.904590 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:49.904828 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:49.905448 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:49.905454 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:49.905469 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0927 22:57:49.905852 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:49.905866 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:49.907097 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:49.907103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:49.909212 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:49.909236 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:49.910530 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:49.910538 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:49.910814 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:49.910886 (+ 72us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 22:57:49.935357 (+ 24471us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42793/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 22:57:49.935842 (+ 485us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.74.190:45827: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42793/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":458,"threads_started":1}
I20260927 22:57:49.938303 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 9685
I20260927 22:57:49.940833 9746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:50.048123 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 9685
2026-09-27T22:57:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (824 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260927 22:57:50.110780 4394 mini_oidc.cc:152] Starting JWKS server
I20260927 22:57:50.113547 4394 webserver.cc:460] Webserver started at https://127.0.0.1:44449/ using document root <none> and password file <none>
I20260927 22:57:50.113883 4394 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 22:57:50.114668 4394 webserver.cc:460] Webserver started at http://127.0.0.1:41357/ using document root <none> and password file <none>
2026-09-27T22:57:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:50Z Disabled control of system clock
I20260927 22:57:50.149577 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:44765
--webserver_interface=127.4.74.190
--webserver_port=0
--builtin_ntp_servers=127.4.74.148:36851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:44765
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44449/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 22:57:50.487408 9788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:50.487905 9788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:50.488070 9788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:50.496909 9788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:50.497054 9788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:50.497094 9788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:50.497219 9788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:50.497306 9788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 22:57:50.497426 9788 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 22:57:50.497521 9788 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44449/jwks.json
I20260927 22:57:50.507022 9788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:44765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:44765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44449/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:50.508482 9788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:50.510310 9788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:50.523371 9793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:50.523392 9796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:50.523464 9794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:50.525045 9788 server_base.cc:1034] running on GCE node
I20260927 22:57:50.526192 9788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:50.528743 9788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:50.530071 9788 hybrid_clock.cc:648] HybridClock initialized: now 1790549870530011 us; error 64 us; skew 500 ppm
I20260927 22:57:50.530613 9788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:57:50.533768 9788 webserver.cc:460] Webserver started at http://127.4.74.190:39557/ using document root <none> and password file <none>
I20260927 22:57:50.534699 9788 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:50.534848 9788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:50.535234 9788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:50.538867 9788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "73b4500938de4528a1f3417bc6831df5"
format_stamp: "Formatted at 2026-09-27 22:57:50 on dist-test-slave-5pd3"
server_key: "76db301cf39fc027ef4c3d96366d5571"
server_key_iv: "1ed9ce1001221e4fc3b73f310929f7df"
server_key_version: "encryptionkey@0"
I20260927 22:57:50.539971 9788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "73b4500938de4528a1f3417bc6831df5"
format_stamp: "Formatted at 2026-09-27 22:57:50 on dist-test-slave-5pd3"
server_key: "76db301cf39fc027ef4c3d96366d5571"
server_key_iv: "1ed9ce1001221e4fc3b73f310929f7df"
server_key_version: "encryptionkey@0"
I20260927 22:57:50.546545 9788 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260927 22:57:50.551184 9802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:50.553074 9788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260927 22:57:50.553334 9788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "73b4500938de4528a1f3417bc6831df5"
format_stamp: "Formatted at 2026-09-27 22:57:50 on dist-test-slave-5pd3"
server_key: "76db301cf39fc027ef4c3d96366d5571"
server_key_iv: "1ed9ce1001221e4fc3b73f310929f7df"
server_key_version: "encryptionkey@0"
I20260927 22:57:50.553620 9788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:50.579622 9788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:50.580767 9788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:50.581144 9788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:50.604415 9788 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:44765
I20260927 22:57:50.604439 9853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:44765 every 8 connection(s)
I20260927 22:57:50.606276 9788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:50.610775 9854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:50.615422 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 9788
I20260927 22:57:50.615790 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:50.616513 4394 external_mini_cluster.cc:1428] Setting key 5cf11a36d9b5ea0dc56617bc1c477f5b
I20260927 22:57:50.622437 9854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5: Bootstrap starting.
I20260927 22:57:50.626185 9854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:50.627512 9854 log.cc:824] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:50.631490 9854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5: No bootstrap required, opened a new log
I20260927 22:57:50.634639 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:50.618827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.4.74.190:44765)
0927 22:57:50.620258 (+ 1431us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:50.620273 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:50.620332 (+ 59us) server_negotiation.cc:407] Connection header received
0927 22:57:50.622759 (+ 2427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:50.622805 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:50.623482 (+ 677us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:50.624064 (+ 582us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:50.624777 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:50.626671 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:50.628720 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:50.629890 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:50.630074 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:50.631462 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:50.631471 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:50.631494 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:50.632358 (+ 864us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:50.632463 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:50.632490 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:50.632604 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:50.632875 (+ 271us) server_negotiation.cc:299] Negotiation successful
0927 22:57:50.633331 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":502,"threads_started":1}
I20260927 22:57:50.639693 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 22:57:50.640471 9854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b4500938de4528a1f3417bc6831df5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 44765 } }
I20260927 22:57:50.640960 9854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:50.641073 9854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b4500938de4528a1f3417bc6831df5, State: Initialized, Role: FOLLOWER
I20260927 22:57:50.641770 9854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b4500938de4528a1f3417bc6831df5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 44765 } }
I20260927 22:57:50.642050 9854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:50.642205 9854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:50.642391 9854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:50.644971 9854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b4500938de4528a1f3417bc6831df5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 44765 } }
I20260927 22:57:50.645476 9854 leader_election.cc:302] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73b4500938de4528a1f3417bc6831df5; no voters:
I20260927 22:57:50.646072 9854 leader_election.cc:288] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:50.646361 9859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:50.647583 9859 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [term 1 LEADER]: Becoming Leader. State: Replica: 73b4500938de4528a1f3417bc6831df5, State: Running, Role: LEADER
I20260927 22:57:50.648442 9854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:50.648463 9859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b4500938de4528a1f3417bc6831df5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 44765 } }
I20260927 22:57:50.652439 9861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73b4500938de4528a1f3417bc6831df5. Latest consensus state: current_term: 1 leader_uuid: "73b4500938de4528a1f3417bc6831df5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b4500938de4528a1f3417bc6831df5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 44765 } } }
I20260927 22:57:50.652896 9861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:50.653509 9860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73b4500938de4528a1f3417bc6831df5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b4500938de4528a1f3417bc6831df5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 44765 } } }
I20260927 22:57:50.653882 9860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:50.654034 9868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:50.660995 9868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:50.672415 9868 catalog_manager.cc:1347] Generated new cluster ID: a3ac83fb5d304a44923c6eaa256aab9d
I20260927 22:57:50.672586 9868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:50.686920 9868 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:50.688578 9868 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:50.697592 9868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73b4500938de4528a1f3417bc6831df5: Generated new TSK 0
I20260927 22:57:50.698750 9868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 22:57:50.739471 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:50.730159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55488 (local address 127.4.74.190:44765)
0927 22:57:50.730391 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:50.730397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:50.730800 (+ 403us) server_negotiation.cc:407] Connection header received
0927 22:57:50.731093 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:50.731100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:50.731204 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:50.731426 (+ 222us) server_negotiation.cc:226] Negotiated authn=JWT
0927 22:57:50.732004 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:50.733165 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:50.734684 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:50.735387 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:50.735526 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:50.736196 (+ 670us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 22:57:50.738750 (+ 2554us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44449/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 22:57:50.739113 (+ 363us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55488: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44449/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":95}
W20260927 22:57:50.739652 9857 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55488: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44449/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260927 22:57:50.741591 9883 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:57:50.729960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:44765 (local address 127.0.0.1:55488)
0927 22:57:50.730669 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:50.730689 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:50.730854 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:50.731454 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:50.731458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:50.731469 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0927 22:57:50.731823 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:50.731833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:50.733343 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:50.733348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:50.734528 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:50.734540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:50.735675 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:50.735682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:50.735837 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:50.735898 (+ 61us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 22:57:50.739020 (+ 3122us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44449/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 22:57:50.741045 (+ 2025us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.74.190:44765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44449/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":1604,"thread_start_us":388,"threads_started":1}
I20260927 22:57:50.742672 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 9788
I20260927 22:57:50.744258 9849 generic_service.cc:196] Checking for leaks (request via RPC)
E20260927 22:57:50.752609 9884 webserver.cc:533] Webserver: sslize failed:
E20260927 22:57:50.752741 9884 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 22:57:50.840525 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 9788
2026-09-27T22:57:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (786 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-27T22:57:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:50Z Disabled control of system clock
I20260927 22:57:50.929942 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:39815
--webserver_interface=127.4.74.190
--webserver_port=0
--builtin_ntp_servers=127.4.74.148:44383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:39815
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/testchainca.cert with env {}
W20260927 22:57:51.284865 9889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:51.285279 9889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:51.285375 9889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:51.292780 9889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:51.292923 9889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:51.292963 9889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:51.293012 9889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:51.293063 9889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 22:57:51.293131 9889 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 22:57:51.293176 9889 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260927 22:57:51.303505 9889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:44383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:39815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:39815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:51.304937 9889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:51.306283 9889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:51.320298 9895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:51.320377 9894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:51.320329 9897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:51.322009 9889 server_base.cc:1034] running on GCE node
I20260927 22:57:51.323004 9889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:51.324721 9889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:51.325980 9889 hybrid_clock.cc:648] HybridClock initialized: now 1790549871325913 us; error 80 us; skew 500 ppm
I20260927 22:57:51.326409 9889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:57:51.329537 9889 webserver.cc:460] Webserver started at http://127.4.74.190:34393/ using document root <none> and password file <none>
I20260927 22:57:51.330408 9889 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:51.330554 9889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:51.330880 9889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:51.334024 9889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "4a40d0105a5c4cf6a5117fc696fb69a5"
format_stamp: "Formatted at 2026-09-27 22:57:51 on dist-test-slave-5pd3"
server_key: "3fba0ba02593796493c64d9cdef3d2de"
server_key_iv: "4ec231972654df78b110aac335686bc9"
server_key_version: "encryptionkey@0"
I20260927 22:57:51.334946 9889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "4a40d0105a5c4cf6a5117fc696fb69a5"
format_stamp: "Formatted at 2026-09-27 22:57:51 on dist-test-slave-5pd3"
server_key: "3fba0ba02593796493c64d9cdef3d2de"
server_key_iv: "4ec231972654df78b110aac335686bc9"
server_key_version: "encryptionkey@0"
I20260927 22:57:51.340821 9889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260927 22:57:51.345170 9903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:51.346971 9889 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 22:57:51.347285 9889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "4a40d0105a5c4cf6a5117fc696fb69a5"
format_stamp: "Formatted at 2026-09-27 22:57:51 on dist-test-slave-5pd3"
server_key: "3fba0ba02593796493c64d9cdef3d2de"
server_key_iv: "4ec231972654df78b110aac335686bc9"
server_key_version: "encryptionkey@0"
I20260927 22:57:51.347543 9889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:51.369199 9889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:51.370107 9889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:51.370412 9889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:51.391534 9889 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:39815
I20260927 22:57:51.391690 9954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:39815 every 8 connection(s)
I20260927 22:57:51.392999 9889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:51.396570 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 9889
I20260927 22:57:51.396816 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:51.397745 4394 external_mini_cluster.cc:1428] Setting key 1590218a0fb9534eb9ec67b6f4d9f8f4
I20260927 22:57:51.398536 9955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:51.413689 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:51.400175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53670 (local address 127.4.74.190:39815)
0927 22:57:51.401454 (+ 1279us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:51.401468 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:51.401518 (+ 50us) server_negotiation.cc:407] Connection header received
0927 22:57:51.404139 (+ 2621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:51.404164 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:51.404761 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:51.405194 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:51.405815 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:51.407233 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:51.408639 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:51.409364 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:51.409572 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:51.410828 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:51.410835 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:51.410854 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:57:51.411505 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:51.411673 (+ 168us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:57:51.411699 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:51.411832 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:51.412062 (+ 230us) server_negotiation.cc:299] Negotiation successful
0927 22:57:51.412412 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":431,"threads_started":1}
I20260927 22:57:51.414793 9955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5: Bootstrap starting.
I20260927 22:57:51.418843 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 22:57:51.419413 9955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:51.420727 9955 log.cc:824] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:51.424341 9955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5: No bootstrap required, opened a new log
I20260927 22:57:51.433869 9955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a40d0105a5c4cf6a5117fc696fb69a5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39815 } }
I20260927 22:57:51.434369 9955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:51.434509 9955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a40d0105a5c4cf6a5117fc696fb69a5, State: Initialized, Role: FOLLOWER
I20260927 22:57:51.435261 9955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a40d0105a5c4cf6a5117fc696fb69a5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39815 } }
I20260927 22:57:51.435492 9955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:51.435712 9955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:51.435920 9955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:51.438410 9955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a40d0105a5c4cf6a5117fc696fb69a5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39815 } }
I20260927 22:57:51.439209 9955 leader_election.cc:302] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a40d0105a5c4cf6a5117fc696fb69a5; no voters:
I20260927 22:57:51.440065 9955 leader_election.cc:288] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:51.440308 9960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:51.441746 9960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [term 1 LEADER]: Becoming Leader. State: Replica: 4a40d0105a5c4cf6a5117fc696fb69a5, State: Running, Role: LEADER
I20260927 22:57:51.442404 9960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a40d0105a5c4cf6a5117fc696fb69a5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39815 } }
I20260927 22:57:51.442911 9955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:51.446403 9961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a40d0105a5c4cf6a5117fc696fb69a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a40d0105a5c4cf6a5117fc696fb69a5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39815 } } }
I20260927 22:57:51.447014 9961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:51.446306 9962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a40d0105a5c4cf6a5117fc696fb69a5. Latest consensus state: current_term: 1 leader_uuid: "4a40d0105a5c4cf6a5117fc696fb69a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a40d0105a5c4cf6a5117fc696fb69a5" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39815 } } }
I20260927 22:57:51.447192 9962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:51.456525 9968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:51.461869 9968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:51.473764 9968 catalog_manager.cc:1347] Generated new cluster ID: 60b976154a0c455da48f78082663d947
I20260927 22:57:51.473932 9968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:51.495057 9968 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:51.496675 9968 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:51.511098 9968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a40d0105a5c4cf6a5117fc696fb69a5: Generated new TSK 0
I20260927 22:57:51.512415 9968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 22:57:52.017712 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:51.529808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53686 (local address 127.4.74.190:39815)
0927 22:57:51.530056 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:51.530063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:51.530442 (+ 379us) server_negotiation.cc:407] Connection header received
0927 22:57:51.530906 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:51.530912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:51.530993 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:51.531158 (+ 165us) server_negotiation.cc:226] Negotiated authn=JWT
0927 22:57:51.531932 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:51.532770 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:51.535011 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:51.535503 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:51.535742 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:51.536503 (+ 761us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 22:57:52.016943 (+480440us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 22:57:52.017405 (+ 462us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53686: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":98}
W20260927 22:57:52.017853 9958 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53686: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260927 22:57:52.018110 9984 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:57:51.529407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39815 (local address 127.0.0.1:53686)
0927 22:57:51.530264 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:51.530291 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:51.530631 (+ 340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:51.531235 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:51.531241 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:51.531256 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0927 22:57:51.531715 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:51.531729 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:51.532943 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:51.532949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:51.534803 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:51.534819 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:51.535804 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:51.535812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:51.536031 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:51.536186 (+ 155us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 22:57:52.017324 (+481138us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 22:57:52.017673 (+ 349us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.74.190:39815: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":420,"threads_started":1}
I20260927 22:57:52.019183 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 9889
I20260927 22:57:52.020699 9950 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:52.129876 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 9889
2026-09-27T22:57:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1276 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (526 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (456 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:53 dist-test-slave-5pd3 krb5kdc[9999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:53 dist-test-slave-5pd3 krb5kdc[9999](info): set up 2 sockets
Sep 27 22:57:53 dist-test-slave-5pd3 krb5kdc[9999](info): commencing operation
krb5kdc: starting...
W20260927 22:57:54.296496 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.007s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:54 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:57:54.553020 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:35805
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:35805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:54.892428 10012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:54.892753 10012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:54.892889 10012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:54.900566 10012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:54.900708 10012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:54.900759 10012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:54.900856 10012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:54.900933 10012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:54.911486 10012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:35805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:35805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:54.913002 10012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:54.914363 10012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:54.929584 10018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:54.929688 10017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:54.929697 10020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:54.930676 10012 server_base.cc:1034] running on GCE node
I20260927 22:57:54.931521 10012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:54.933173 10012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:54.934443 10012 hybrid_clock.cc:648] HybridClock initialized: now 1790549874934359 us; error 72 us; skew 500 ppm
I20260927 22:57:54.934837 10012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:54 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549874, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:54.938216 10012 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:57:54.940089 10012 webserver.cc:460] Webserver started at http://127.4.74.190:33357/ using document root <none> and password file <none>
I20260927 22:57:54.940822 10012 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:54.940930 10012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:54.941190 10012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:54.944430 10012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "68f3d8bcf0a34176b7cd14fbfda26f10"
format_stamp: "Formatted at 2026-09-27 22:57:54 on dist-test-slave-5pd3"
server_key: "74f65461bdd8717121cc493226994f46"
server_key_iv: "75dcd9a2d1be1cbaaf07bf61dbcc7c5d"
server_key_version: "encryptionkey@0"
I20260927 22:57:54.945197 10012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "68f3d8bcf0a34176b7cd14fbfda26f10"
format_stamp: "Formatted at 2026-09-27 22:57:54 on dist-test-slave-5pd3"
server_key: "74f65461bdd8717121cc493226994f46"
server_key_iv: "75dcd9a2d1be1cbaaf07bf61dbcc7c5d"
server_key_version: "encryptionkey@0"
I20260927 22:57:54.952025 10012 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 22:57:54.956189 10027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:54.957990 10012 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 22:57:54.958192 10012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "68f3d8bcf0a34176b7cd14fbfda26f10"
format_stamp: "Formatted at 2026-09-27 22:57:54 on dist-test-slave-5pd3"
server_key: "74f65461bdd8717121cc493226994f46"
server_key_iv: "75dcd9a2d1be1cbaaf07bf61dbcc7c5d"
server_key_version: "encryptionkey@0"
I20260927 22:57:54.958391 10012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:54.977245 10012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:54.981772 10012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:54.982025 10012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:55.006549 10012 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:35805
I20260927 22:57:55.006680 10078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:35805 every 8 connection(s)
I20260927 22:57:55.008178 10012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:55.009729 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 10012
I20260927 22:57:55.010031 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:55.010780 4394 external_mini_cluster.cc:1428] Setting key 5edc7e4b97f25b5b0be663180cb3656c
I20260927 22:57:55.023099 10079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 22:57:55 dist-test-slave-5pd3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:55.043491 10079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10: Bootstrap starting.
I20260927 22:57:55.047376 10079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:55.048774 10079 log.cc:824] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:55.052160 10079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10: No bootstrap required, opened a new log
I20260927 22:57:55.054193 10082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:55.023253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46902 (local address 127.4.74.190:35805)
0927 22:57:55.025121 (+ 1868us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:55.025144 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:55.025203 (+ 59us) server_negotiation.cc:407] Connection header received
0927 22:57:55.028410 (+ 3207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:55.028454 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:55.029216 (+ 762us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:55.029737 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:55.031859 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:55.033587 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:55.036492 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:55.037412 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:55.037590 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:55.043828 (+ 6238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:55.043843 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:55.043881 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:55.043950 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:55.048809 (+ 4859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:55.049866 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:55.049875 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:55.049884 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:55.050123 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:55.050611 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:55.050615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:55.050617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:55.051336 (+ 719us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:55.051753 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:55.052256 (+ 503us) server_negotiation.cc:299] Negotiation successful
0927 22:57:55.052857 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1530,"thread_start_us":703,"threads_started":1}
I20260927 22:57:55.062018 10079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3d8bcf0a34176b7cd14fbfda26f10" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35805 } }
I20260927 22:57:55.062567 10079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:55.062734 10079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f3d8bcf0a34176b7cd14fbfda26f10, State: Initialized, Role: FOLLOWER
I20260927 22:57:55.063656 10079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3d8bcf0a34176b7cd14fbfda26f10" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35805 } }
I20260927 22:57:55.063941 10079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:55.064106 10079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:55.064327 10079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:55.066860 10079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3d8bcf0a34176b7cd14fbfda26f10" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35805 } }
I20260927 22:57:55.067497 10079 leader_election.cc:302] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68f3d8bcf0a34176b7cd14fbfda26f10; no voters:
I20260927 22:57:55.068455 10079 leader_election.cc:288] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:55.068713 10084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:55.070328 10084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [term 1 LEADER]: Becoming Leader. State: Replica: 68f3d8bcf0a34176b7cd14fbfda26f10, State: Running, Role: LEADER
I20260927 22:57:55.070916 10079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:55.071130 10084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3d8bcf0a34176b7cd14fbfda26f10" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35805 } }
I20260927 22:57:55.074892 10086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68f3d8bcf0a34176b7cd14fbfda26f10. Latest consensus state: current_term: 1 leader_uuid: "68f3d8bcf0a34176b7cd14fbfda26f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3d8bcf0a34176b7cd14fbfda26f10" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35805 } } }
I20260927 22:57:55.075798 10086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:55.075520 10085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68f3d8bcf0a34176b7cd14fbfda26f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f3d8bcf0a34176b7cd14fbfda26f10" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35805 } } }
I20260927 22:57:55.076086 10085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:55.078043 10094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:55.088898 10094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:55.102263 10094 catalog_manager.cc:1347] Generated new cluster ID: 3b32d79494f449a487e82d02b2866a4f
I20260927 22:57:55.102455 10094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:55.129880 10094 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:55.131630 10094 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:55.152352 10094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68f3d8bcf0a34176b7cd14fbfda26f10: Generated new TSK 0
I20260927 22:57:55.153337 10094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:57:55.224304 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35805
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:55.573935 10105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:55.574342 10105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:55.574530 10105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:55.582263 10105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:55.582397 10105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:55.582582 10105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:57:55.593384 10105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:55.594831 10105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:55.596289 10105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:55.611799 10111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:55.611914 10113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:55.611970 10110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:55.613148 10105 server_base.cc:1034] running on GCE node
I20260927 22:57:55.614094 10105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:55.615584 10105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:55.616846 10105 hybrid_clock.cc:648] HybridClock initialized: now 1790549875616787 us; error 52 us; skew 500 ppm
I20260927 22:57:55.617302 10105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:55 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549875, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:55.621249 10105 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:57:55.623162 10105 webserver.cc:460] Webserver started at http://127.4.74.129:42543/ using document root <none> and password file <none>
I20260927 22:57:55.624135 10105 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:55.624260 10105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:55.624676 10105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:55.628615 10105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "36f8f2ee004e4b018c3d9a4e5d5b3704"
format_stamp: "Formatted at 2026-09-27 22:57:55 on dist-test-slave-5pd3"
server_key: "bbb90126260cde327f11fa444a63d25d"
server_key_iv: "b899666fb6c01d38bee4d8571e78c692"
server_key_version: "encryptionkey@0"
I20260927 22:57:55.629653 10105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "36f8f2ee004e4b018c3d9a4e5d5b3704"
format_stamp: "Formatted at 2026-09-27 22:57:55 on dist-test-slave-5pd3"
server_key: "bbb90126260cde327f11fa444a63d25d"
server_key_iv: "b899666fb6c01d38bee4d8571e78c692"
server_key_version: "encryptionkey@0"
I20260927 22:57:55.636468 10105 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260927 22:57:55.640957 10120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:55.642705 10105 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:57:55.642910 10105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "36f8f2ee004e4b018c3d9a4e5d5b3704"
format_stamp: "Formatted at 2026-09-27 22:57:55 on dist-test-slave-5pd3"
server_key: "bbb90126260cde327f11fa444a63d25d"
server_key_iv: "b899666fb6c01d38bee4d8571e78c692"
server_key_version: "encryptionkey@0"
I20260927 22:57:55.643229 10105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:55.658715 10105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:55.664448 10105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:55.664832 10105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:55.666076 10105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:55.667845 10105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:55.667932 10105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:55.668095 10105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:55.668190 10105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:55.709853 10105 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:46781
I20260927 22:57:55.709918 10232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:46781 every 8 connection(s)
I20260927 22:57:55.711462 10105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:57:55.717857 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 10105
I20260927 22:57:55.718139 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:57:55.718856 4394 external_mini_cluster.cc:1428] Setting key 91932b0c0c26f418553bd06e6049f877
Sep 27 22:57:55 dist-test-slave-5pd3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549875, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:55.735877 10235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:55.714544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:44041 (local address 127.4.74.190:35805)
0927 22:57:55.715060 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:55.715065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:55.716313 (+ 1248us) server_negotiation.cc:407] Connection header received
0927 22:57:55.718372 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:55.718377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:55.718475 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:55.718596 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:55.720517 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:55.721430 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:55.723223 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:55.724121 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:55.724343 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:55.728185 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:55.728194 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:55.728201 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:55.728263 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:55.731541 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:55.732750 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:55.732755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:55.732758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:55.732934 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:55.733612 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:55.733617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:55.733619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:55.733974 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:55.734123 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:55.735226 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0927 22:57:55.735473 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":316,"threads_started":1}
I20260927 22:57:55.737495 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:55.715171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35805 (local address 127.4.74.129:44041)
0927 22:57:55.716118 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:55.716172 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:55.718017 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:55.718862 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:55.718870 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:55.719534 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:55.720339 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:55.720357 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:55.721600 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:55.721605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:55.722983 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:55.722999 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:55.724397 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:55.724403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:55.724610 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:55.725437 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:55.725459 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:55.727866 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:55.731837 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:55.731850 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:55.731872 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:55.732563 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:55.733072 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:55.733079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:55.733085 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:55.733360 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:55.734151 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:55.734162 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:55.734697 (+ 535us) client_negotiation.cc:769] Sending connection context
0927 22:57:55.735063 (+ 366us) client_negotiation.cc:240] Negotiation successful
0927 22:57:55.735725 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":413,"threads_started":1}
I20260927 22:57:55.739621 10233 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35805
I20260927 22:57:55.740099 10233 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:55.741132 10233 heartbeater.cc:511] Master 127.4.74.190:35805 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 22:57:55.744113 10044 ts_manager.cc:194] Registered new tserver with Master: 36f8f2ee004e4b018c3d9a4e5d5b3704 (127.4.74.129:46781)
I20260927 22:57:55.745982 10044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:44041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:57:55.774986 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35805
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:56.132468 10239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:56.132788 10239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:56.132946 10239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:56.140775 10239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:56.140905 10239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:56.141088 10239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:57:56.151612 10239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:56.153319 10239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:56.154894 10239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:56.170120 10244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:56.170074 10245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:56.170231 10247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:56.171607 10239 server_base.cc:1034] running on GCE node
I20260927 22:57:56.172600 10239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:56.173900 10239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:56.175189 10239 hybrid_clock.cc:648] HybridClock initialized: now 1790549876175089 us; error 82 us; skew 500 ppm
I20260927 22:57:56.175779 10239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:56 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549876, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:56.180034 10239 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:57:56.182101 10239 webserver.cc:460] Webserver started at http://127.4.74.130:36123/ using document root <none> and password file <none>
I20260927 22:57:56.183017 10239 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:56.183174 10239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:56.183624 10239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:56.187773 10239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "debfd43f11544f8b9da2502600630bf8"
format_stamp: "Formatted at 2026-09-27 22:57:56 on dist-test-slave-5pd3"
server_key: "dc1d555245182c1c265172e4262d792c"
server_key_iv: "b1581288f35ef92ec271f6cb48efbf9f"
server_key_version: "encryptionkey@0"
I20260927 22:57:56.188871 10239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "debfd43f11544f8b9da2502600630bf8"
format_stamp: "Formatted at 2026-09-27 22:57:56 on dist-test-slave-5pd3"
server_key: "dc1d555245182c1c265172e4262d792c"
server_key_iv: "b1581288f35ef92ec271f6cb48efbf9f"
server_key_version: "encryptionkey@0"
I20260927 22:57:56.196453 10239 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260927 22:57:56.201721 10254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:56.203946 10239 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.001s
I20260927 22:57:56.204242 10239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "debfd43f11544f8b9da2502600630bf8"
format_stamp: "Formatted at 2026-09-27 22:57:56 on dist-test-slave-5pd3"
server_key: "dc1d555245182c1c265172e4262d792c"
server_key_iv: "b1581288f35ef92ec271f6cb48efbf9f"
server_key_version: "encryptionkey@0"
I20260927 22:57:56.204519 10239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:56.221164 10239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:56.226850 10239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:56.227288 10239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:56.228672 10239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:56.230335 10239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:56.230427 10239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:56.230600 10239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:56.230661 10239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:56.271247 10239 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:34177
I20260927 22:57:56.271399 10366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:34177 every 8 connection(s)
I20260927 22:57:56.273082 10239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:57:56.273558 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 10239
I20260927 22:57:56.273751 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:57:56.274403 4394 external_mini_cluster.cc:1428] Setting key f6377f786f3206360c7b58ce0c075306
Sep 27 22:57:56 dist-test-slave-5pd3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549876, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 22:57:56.298365 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:56.276651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:40111 (local address 127.4.74.190:35805)
0927 22:57:56.277206 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:56.277213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:56.278663 (+ 1450us) server_negotiation.cc:407] Connection header received
0927 22:57:56.281816 (+ 3153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:56.281821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:56.281919 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:56.282059 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:56.284670 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:56.285412 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:56.287109 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:56.287636 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:56.287779 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:56.291877 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:56.291883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:56.291886 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:56.291923 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:56.294946 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:56.295921 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:56.295925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:56.295928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:56.296086 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:56.296522 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:56.296529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:56.296537 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:56.296821 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:56.296987 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:56.297708 (+ 721us) server_negotiation.cc:299] Negotiation successful
0927 22:57:56.297961 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":277,"threads_started":1}
I20260927 22:57:56.299484 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:56.277352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35805 (local address 127.4.74.130:40111)
0927 22:57:56.278434 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:56.278499 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:56.281403 (+ 2904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:56.282420 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:56.282436 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:56.283377 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:56.284390 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:56.284420 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:56.285594 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:56.285600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:56.286934 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:56.286946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:56.287884 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:56.287893 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:56.288210 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:56.289126 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:56.289153 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:56.291556 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:56.295150 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:56.295161 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:56.295187 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:56.295737 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:56.296211 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:56.296217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:56.296221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:56.296405 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:56.296951 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:56.296960 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:56.297339 (+ 379us) client_negotiation.cc:769] Sending connection context
0927 22:57:56.297609 (+ 270us) client_negotiation.cc:240] Negotiation successful
0927 22:57:56.298184 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":376,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
I20260927 22:57:56.301472 10367 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35805
I20260927 22:57:56.301956 10367 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:56.302999 10367 heartbeater.cc:511] Master 127.4.74.190:35805 requested a full tablet report, sending...
I20260927 22:57:56.305266 10044 ts_manager.cc:194] Registered new tserver with Master: debfd43f11544f8b9da2502600630bf8 (127.4.74.130:34177)
I20260927 22:57:56.306610 10044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:40111
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:57:56.332422 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35805
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:56.744974 10373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:56.745431 10373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:56.745615 10373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260927 22:57:56.750113 10233 heartbeater.cc:503] Master 127.4.74.190:35805 was elected leader, sending a full tablet report...
W20260927 22:57:56.754634 10373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:56.754837 10373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:56.755039 10373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:57:56.766683 10373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:56.768577 10373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:56.770275 10373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:56.787611 10378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:56.787612 10379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:56.787812 10381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:56.788712 10373 server_base.cc:1034] running on GCE node
I20260927 22:57:56.789480 10373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:56.790652 10373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:56.791898 10373 hybrid_clock.cc:648] HybridClock initialized: now 1790549876791832 us; error 74 us; skew 500 ppm
I20260927 22:57:56.792268 10373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:56 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549876, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:56.796443 10373 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:57:56.798503 10373 webserver.cc:460] Webserver started at http://127.4.74.131:43411/ using document root <none> and password file <none>
I20260927 22:57:56.799438 10373 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:56.799656 10373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:56.800047 10373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:56.803819 10373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "941a6b2c556141a08bec207e4078e001"
format_stamp: "Formatted at 2026-09-27 22:57:56 on dist-test-slave-5pd3"
server_key: "e2112315e5ba2b103a35fcbf4ecc6552"
server_key_iv: "b2e0c5a313ac42e82d40f8c7c5b62344"
server_key_version: "encryptionkey@0"
I20260927 22:57:56.805694 10373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "941a6b2c556141a08bec207e4078e001"
format_stamp: "Formatted at 2026-09-27 22:57:56 on dist-test-slave-5pd3"
server_key: "e2112315e5ba2b103a35fcbf4ecc6552"
server_key_iv: "b2e0c5a313ac42e82d40f8c7c5b62344"
server_key_version: "encryptionkey@0"
I20260927 22:57:56.815382 10373 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260927 22:57:56.821058 10388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:56.823349 10373 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260927 22:57:56.823727 10373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "941a6b2c556141a08bec207e4078e001"
format_stamp: "Formatted at 2026-09-27 22:57:56 on dist-test-slave-5pd3"
server_key: "e2112315e5ba2b103a35fcbf4ecc6552"
server_key_iv: "b2e0c5a313ac42e82d40f8c7c5b62344"
server_key_version: "encryptionkey@0"
I20260927 22:57:56.824064 10373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:56.847075 10373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:56.852329 10373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:56.852669 10373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:56.853907 10373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:57:56.855710 10373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:57:56.855823 10373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:56.855986 10373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:57:56.856058 10373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:56.898933 10373 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:43483
I20260927 22:57:56.899040 10500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:43483 every 8 connection(s)
I20260927 22:57:56.900493 10373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:57:56.910557 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 10373
I20260927 22:57:56.910732 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:57:56.911276 4394 external_mini_cluster.cc:1428] Setting key c83b093fcf90013a101fd69564e64f78
Sep 27 22:57:56 dist-test-slave-5pd3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549876, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:57:56.928421 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:56.904122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:46911 (local address 127.4.74.190:35805)
0927 22:57:56.904747 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:56.904752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:56.905745 (+ 993us) server_negotiation.cc:407] Connection header received
0927 22:57:56.908348 (+ 2603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:56.908355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:56.908479 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:56.908619 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:56.910923 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:56.911920 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:56.914851 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:56.915597 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:56.915739 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:56.921006 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:56.921013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:56.921016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:56.921064 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:56.924141 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:56.925164 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:56.925173 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:56.925178 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:56.925318 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:56.925824 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:56.925832 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:56.925836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:56.926202 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:56.926387 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:56.927713 (+ 1326us) server_negotiation.cc:299] Negotiation successful
0927 22:57:56.928001 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":357,"threads_started":1}
I20260927 22:57:56.929821 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:56.904556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35805 (local address 127.4.74.131:46911)
0927 22:57:56.905510 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0927 22:57:56.905572 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 22:57:56.907919 (+ 2347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:57:56.908988 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:57:56.909022 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:57:56.909776 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:57:56.910624 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:56.910674 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:56.912273 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:56.912282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:56.914622 (+ 2340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:57:56.914647 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:57:56.915910 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:57:56.915917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:57:56.916208 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:56.917171 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:57:56.917202 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:57:56.920617 (+ 3415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:57:56.924385 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:56.924396 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:56.924417 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:56.925015 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:56.925450 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:57:56.925462 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:57:56.925465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:57:56.925694 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:57:56.926515 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:57:56.926528 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:57:56.927051 (+ 523us) client_negotiation.cc:769] Sending connection context
0927 22:57:56.927445 (+ 394us) client_negotiation.cc:240] Negotiation successful
0927 22:57:56.928220 (+ 775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":354,"threads_started":1}
I20260927 22:57:56.931655 10501 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35805
I20260927 22:57:56.932120 10501 heartbeater.cc:464] Registering TS with master...
I20260927 22:57:56.933151 10501 heartbeater.cc:511] Master 127.4.74.190:35805 requested a full tablet report, sending...
I20260927 22:57:56.935359 10044 ts_manager.cc:194] Registered new tserver with Master: 941a6b2c556141a08bec207e4078e001 (127.4.74.131:43483)
I20260927 22:57:56.936388 10044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:46911
I20260927 22:57:56.942982 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/test-admin.keytab.
Sep 27 22:57:56 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:56.979740 4394 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260927 22:57:56.987475 10509 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:57:56.984255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.4.74.129:46781)
0927 22:57:56.985085 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:56.985093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:56.985124 (+ 31us) server_negotiation.cc:407] Connection header received
0927 22:57:56.985223 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:56.985230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:56.985466 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:56.985616 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:56.986887 (+ 1271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34722: BlockingRecv error: recv got EOF from 127.0.0.1:34722 (error 108)
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":436,"threads_started":1}
W20260927 22:57:56.987805 4394 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5cc' of type FILE
Sep 27 22:57:56 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:56.989959 4394 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 22:57:56 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:56.991689 4394 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 22:57:56 dist-test-slave-5pd3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:57.010172 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:56.993922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34728 (local address 127.4.74.129:46781)
0927 22:57:56.994191 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:56.994201 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:56.994664 (+ 463us) server_negotiation.cc:407] Connection header received
0927 22:57:56.994889 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:56.994895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:56.994976 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:56.995089 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:56.996247 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:56.997254 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:56.998507 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:56.999203 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:56.999364 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:57.003587 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:57.003612 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:57.003632 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:57.003680 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:57.006952 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.007767 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.007777 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.007782 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.007919 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.008340 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.008345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.008354 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.008753 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:57.009076 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:57.009571 (+ 495us) server_negotiation.cc:299] Negotiation successful
0927 22:57:57.009859 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260927 22:57:57.018664 10509 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:57:57.016163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.4.74.129:46781)
0927 22:57:57.016434 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:57.016442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:57.016772 (+ 330us) server_negotiation.cc:407] Connection header received
0927 22:57:57.016922 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:57.016927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:57.017046 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:57.017385 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:57.018351 (+ 966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34736: BlockingRecv error: recv got EOF from 127.0.0.1:34736 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260927 22:57:57.018790 4394 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5cc' of type FILE
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:57.020784 4394 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:57.022320 4394 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:57.039255 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:57.024256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.4.74.129:46781)
0927 22:57:57.024546 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:57.024553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:57.024789 (+ 236us) server_negotiation.cc:407] Connection header received
0927 22:57:57.025007 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:57.025012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:57.025121 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:57.025229 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:57.026281 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:57.026999 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:57.028365 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:57.028878 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:57.029015 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:57.033440 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:57.033445 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:57.033449 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:57.033487 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:57.036376 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.037267 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.037271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.037274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.037438 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.037883 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.037887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.037889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.038227 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:57.038437 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:57.038753 (+ 316us) server_negotiation.cc:299] Negotiation successful
0927 22:57:57.038966 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260927 22:57:57.045149 10509 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:57:57.043107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.4.74.129:46781)
0927 22:57:57.043370 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:57.043385 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:57.043542 (+ 157us) server_negotiation.cc:407] Connection header received
0927 22:57:57.043798 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:57.043802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:57.043899 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:57.043995 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:57.044848 (+ 853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34754: BlockingRecv error: recv got EOF from 127.0.0.1:34754 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260927 22:57:57.045255 4394 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790549821755302-4394-0/krb5kdc/krb5cc' of type FILE
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:57.047191 4394 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:57.049046 4394 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:57:57.066371 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:57.050805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.4.74.129:46781)
0927 22:57:57.051076 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:57.051085 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:57.051247 (+ 162us) server_negotiation.cc:407] Connection header received
0927 22:57:57.051450 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:57.051455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:57.051645 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:57.051752 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:57.052817 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:57.053692 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:57.054977 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:57.055509 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:57.055704 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:57.059686 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:57.059692 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:57.059696 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:57.059751 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:57.063044 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.064013 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.064018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.064021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.064227 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.064706 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.064711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.064713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.065080 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:57.065376 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:57.065676 (+ 300us) server_negotiation.cc:299] Negotiation successful
0927 22:57:57.065983 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260927 22:57:57.068897 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 10105
I20260927 22:57:57.084609 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:57.069772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.4.74.129:46781)
0927 22:57:57.070074 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:57.070082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:57.070363 (+ 281us) server_negotiation.cc:407] Connection header received
0927 22:57:57.070528 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:57.070535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:57.070618 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:57.070772 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:57.071940 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:57.072726 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:57.074178 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:57.074701 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:57.074826 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:57.077402 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:57.077408 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:57.077412 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:57.077457 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:57.081218 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.082170 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.082178 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.082185 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.082361 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.082806 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.082822 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.082826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.083186 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:57.083500 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:57.083909 (+ 409us) server_negotiation.cc:299] Negotiation successful
0927 22:57:57.084181 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260927 22:57:57.085486 10228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:57.176546 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 10105
I20260927 22:57:57.193930 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 10239
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:57:57.214260 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:57.195128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48804 (local address 127.4.74.130:34177)
0927 22:57:57.195736 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:57.195744 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:57.195776 (+ 32us) server_negotiation.cc:407] Connection header received
0927 22:57:57.195848 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:57.195855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:57.196080 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:57.196248 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:57.197729 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:57.198780 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:57.200681 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:57.201462 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:57.201623 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:57.206909 (+ 5286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:57.206927 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:57.206949 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:57.206999 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:57.210711 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.211858 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.211870 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.211877 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.212044 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.212533 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.212537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.212539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.212917 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:57.213166 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:57.213566 (+ 400us) server_negotiation.cc:299] Negotiation successful
0927 22:57:57.213858 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":301,"threads_started":1}
I20260927 22:57:57.216230 10362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:57.311264 10367 heartbeater.cc:503] Master 127.4.74.190:35805 was elected leader, sending a full tablet report...
I20260927 22:57:57.313038 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 10239
W20260927 22:57:57.330917 10029 connection.cc:535] server connection from 127.4.74.130:40111 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 22:57:57.331295 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 10373
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:57:57.351173 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:57.332503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.4.74.131:43483)
0927 22:57:57.333252 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:57.333261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:57.333301 (+ 40us) server_negotiation.cc:407] Connection header received
0927 22:57:57.333393 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:57.333401 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:57.333680 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:57.333859 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:57.335116 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:57.336335 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:57.337811 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:57.338609 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:57.338759 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:57.343887 (+ 5128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:57.343910 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:57.343930 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:57.343988 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:57.347673 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.348773 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.348782 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.348785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.348996 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:57.349465 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:57.349470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:57.349472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:57.349942 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:57.350229 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:57.350540 (+ 311us) server_negotiation.cc:299] Negotiation successful
0927 22:57:57.350799 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":384,"threads_started":1}
I20260927 22:57:57.353328 10496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:57.442876 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 10373
I20260927 22:57:57.461005 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 10012
I20260927 22:57:57.462345 10074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:57:57.571146 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 10012
2026-09-27T22:57:57Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4460 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[10529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[10529](info): set up 2 sockets
Sep 27 22:57:57 dist-test-slave-5pd3 krb5kdc[10529](info): commencing operation
krb5kdc: starting...
W20260927 22:57:58.705220 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:57:58 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:57:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:57:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.190.keytab.
Entry for principal oryx/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.190.keytab.
I20260927 22:57:58.981283 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:34435
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.190.keytab
--principal=oryx/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:34435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:57:59.401615 10542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:57:59.401952 10542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:57:59.402107 10542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:57:59.409780 10542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:57:59.409912 10542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:57:59.409960 10542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:57:59.410086 10542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:57:59.410169 10542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:57:59.420351 10542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:34435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.190.keytab
--principal=oryx/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:34435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:57:59.421872 10542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:57:59.423347 10542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:57:59.436810 10550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:59.436913 10548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:57:59.436844 10547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:57:59.437880 10542 server_base.cc:1034] running on GCE node
I20260927 22:57:59.438954 10542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:57:59.440961 10542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:57:59.442260 10542 hybrid_clock.cc:648] HybridClock initialized: now 1790549879442162 us; error 75 us; skew 500 ppm
I20260927 22:57:59.442848 10542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:57:59 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549879, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:57:59.446871 10542 init.cc:377] Logged in from keytab as oryx/127.4.74.190@KRBTEST.COM (short username oryx)
I20260927 22:57:59.448928 10542 webserver.cc:460] Webserver started at http://127.4.74.190:36255/ using document root <none> and password file <none>
I20260927 22:57:59.449839 10542 fs_manager.cc:359] Metadata directory not provided
I20260927 22:57:59.449967 10542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:57:59.450282 10542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:57:59.454069 10542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "f5325a6273cb4a218f13ac8cb7dd5537"
format_stamp: "Formatted at 2026-09-27 22:57:59 on dist-test-slave-5pd3"
server_key: "ca7a0780723d7b08446e43946dffb821"
server_key_iv: "5feb580239493d643517f9a9f2ab5eee"
server_key_version: "encryptionkey@0"
I20260927 22:57:59.455228 10542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "f5325a6273cb4a218f13ac8cb7dd5537"
format_stamp: "Formatted at 2026-09-27 22:57:59 on dist-test-slave-5pd3"
server_key: "ca7a0780723d7b08446e43946dffb821"
server_key_iv: "5feb580239493d643517f9a9f2ab5eee"
server_key_version: "encryptionkey@0"
I20260927 22:57:59.462491 10542 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260927 22:57:59.467365 10557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:57:59.469659 10542 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:57:59.469934 10542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "f5325a6273cb4a218f13ac8cb7dd5537"
format_stamp: "Formatted at 2026-09-27 22:57:59 on dist-test-slave-5pd3"
server_key: "ca7a0780723d7b08446e43946dffb821"
server_key_iv: "5feb580239493d643517f9a9f2ab5eee"
server_key_version: "encryptionkey@0"
I20260927 22:57:59.470216 10542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:57:59.493366 10542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:57:59.499392 10542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:57:59.499836 10542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:57:59.524278 10542 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:34435
I20260927 22:57:59.524318 10608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:34435 every 8 connection(s)
I20260927 22:57:59.526093 10542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:57:59.530445 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 10542
I20260927 22:57:59.530687 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:57:59.530881 10609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:57:59.531497 4394 external_mini_cluster.cc:1428] Setting key e0502daa581751226e4469be47d5920b
I20260927 22:57:59.546005 10609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537: Bootstrap starting.
Sep 27 22:57:59 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:57:59.550205 10609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537: Neither blocks nor log segments found. Creating new log.
I20260927 22:57:59.551656 10609 log.cc:824] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537: Log is configured to *not* fsync() on all Append() calls
I20260927 22:57:59.555912 10609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537: No bootstrap required, opened a new log
I20260927 22:57:59.560694 10612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:57:59.534073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.4.74.190:34435)
0927 22:57:59.535086 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0927 22:57:59.535103 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 22:57:59.535158 (+ 55us) server_negotiation.cc:407] Connection header received
0927 22:57:59.537621 (+ 2463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:57:59.537650 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:57:59.538189 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:57:59.538644 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:57:59.540540 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:59.542127 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:59.544138 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:57:59.545056 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:57:59.545241 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:57:59.550596 (+ 5355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:57:59.550621 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:57:59.550652 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:57:59.550730 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:57:59.555113 (+ 4383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:59.556325 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:59.556334 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:59.556343 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:59.556623 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:57:59.557045 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:57:59.557051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:57:59.557054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:57:59.557827 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:57:59.558170 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0927 22:57:59.558655 (+ 485us) server_negotiation.cc:299] Negotiation successful
0927 22:57:59.559156 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":495,"threads_started":1}
I20260927 22:57:59.565498 10609 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } }
I20260927 22:57:59.565860 10609 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:57:59.565935 10609 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5325a6273cb4a218f13ac8cb7dd5537, State: Initialized, Role: FOLLOWER
I20260927 22:57:59.566466 10609 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } }
I20260927 22:57:59.566668 10609 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:57:59.566766 10609 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:57:59.566888 10609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:57:59.569304 10609 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } }
I20260927 22:57:59.569859 10609 leader_election.cc:302] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5325a6273cb4a218f13ac8cb7dd5537; no voters:
I20260927 22:57:59.570668 10609 leader_election.cc:288] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:57:59.570839 10614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:57:59.571758 10614 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 1 LEADER]: Becoming Leader. State: Replica: f5325a6273cb4a218f13ac8cb7dd5537, State: Running, Role: LEADER
I20260927 22:57:59.572441 10614 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } }
I20260927 22:57:59.573132 10609 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:57:59.576248 10615 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } }
I20260927 22:57:59.577440 10615 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:59.577015 10616 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5325a6273cb4a218f13ac8cb7dd5537. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } }
I20260927 22:57:59.578137 10616 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
I20260927 22:57:59.578980 10620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:57:59.587883 10620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:57:59.599504 10620 catalog_manager.cc:1347] Generated new cluster ID: 24c7cb2cc7914dd0be03d66f9624828c
I20260927 22:57:59.599738 10620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:57:59.615370 10620 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:57:59.616856 10620 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:57:59.626719 10620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537: Generated new TSK 0
I20260927 22:57:59.627803 10620 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab.
Entry for principal oryx/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab.
I20260927 22:57:59.700834 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34435
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab
--principal=oryx/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:00.103816 10635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:00.104184 10635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:00.104364 10635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:00.114710 10635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:00.114881 10635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:00.115202 10635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:58:00.125862 10635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab
--principal=oryx/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:00.127334 10635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:00.128844 10635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:00.144913 10640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:00.144914 10641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:00.144809 10643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:00.145375 10635 server_base.cc:1034] running on GCE node
I20260927 22:58:00.146526 10635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:00.147811 10635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:00.149070 10635 hybrid_clock.cc:648] HybridClock initialized: now 1790549880148992 us; error 81 us; skew 500 ppm
I20260927 22:58:00.149441 10635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:00 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549880, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:00.153200 10635 init.cc:377] Logged in from keytab as oryx/127.4.74.129@KRBTEST.COM (short username oryx)
I20260927 22:58:00.154994 10635 webserver.cc:460] Webserver started at http://127.4.74.129:33111/ using document root <none> and password file <none>
I20260927 22:58:00.155853 10635 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:00.156004 10635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:00.156319 10635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:00.159425 10635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "777d65cebeda4de0baf50caf516ee3f9"
format_stamp: "Formatted at 2026-09-27 22:58:00 on dist-test-slave-5pd3"
server_key: "5b9c6eeb7bf725ec3f14a6f4fb197431"
server_key_iv: "dd4b2308d379435bb6741af84df51824"
server_key_version: "encryptionkey@0"
I20260927 22:58:00.160316 10635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "777d65cebeda4de0baf50caf516ee3f9"
format_stamp: "Formatted at 2026-09-27 22:58:00 on dist-test-slave-5pd3"
server_key: "5b9c6eeb7bf725ec3f14a6f4fb197431"
server_key_iv: "dd4b2308d379435bb6741af84df51824"
server_key_version: "encryptionkey@0"
I20260927 22:58:00.166214 10635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260927 22:58:00.170717 10650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:00.172585 10635 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 22:58:00.172794 10635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "777d65cebeda4de0baf50caf516ee3f9"
format_stamp: "Formatted at 2026-09-27 22:58:00 on dist-test-slave-5pd3"
server_key: "5b9c6eeb7bf725ec3f14a6f4fb197431"
server_key_iv: "dd4b2308d379435bb6741af84df51824"
server_key_version: "encryptionkey@0"
I20260927 22:58:00.173039 10635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:00.196911 10635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:00.202088 10635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:00.202467 10635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:00.203614 10635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:00.205225 10635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:00.205338 10635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:00.205478 10635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:00.205545 10635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:00.246090 10635 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:39233
I20260927 22:58:00.246193 10762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:39233 every 8 connection(s)
I20260927 22:58:00.247514 10635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:58:00.252542 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 10635
I20260927 22:58:00.252791 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:58:00.253726 4394 external_mini_cluster.cc:1428] Setting key 71b644c151dd0fc6153e8cded1335e1b
Sep 27 22:58:00 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549880, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:00.278649 10765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:00.250880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:45275 (local address 127.4.74.190:34435)
0927 22:58:00.251479 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:00.251486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:00.254583 (+ 3097us) server_negotiation.cc:407] Connection header received
0927 22:58:00.256178 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:00.256182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:00.256265 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:00.256401 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:00.258641 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:00.259454 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:00.261479 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:00.261971 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:00.262140 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:00.269713 (+ 7573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:00.269725 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:00.269735 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:00.269818 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:00.274180 (+ 4362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:00.275379 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:00.275388 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:00.275394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:00.275650 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:00.276205 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:00.276213 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:00.276219 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:00.276730 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:00.276892 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:00.277811 (+ 919us) server_negotiation.cc:299] Negotiation successful
0927 22:58:00.278116 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":285,"threads_started":1}
I20260927 22:58:00.279677 10766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:00.251464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.129:45275)
0927 22:58:00.252698 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:00.252780 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:00.255531 (+ 2751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:00.256755 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:00.256768 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:00.257596 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:00.258373 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:00.258401 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:00.259693 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:00.259698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:00.261267 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:00.261288 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:00.262295 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:00.262303 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:00.264262 (+ 1959us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:00.265402 (+ 1140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:00.265436 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:00.269284 (+ 3848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:00.274464 (+ 5180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:00.274474 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:00.274491 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:00.275173 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:00.275783 (+ 610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:00.275787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:00.275789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:00.276051 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:00.276856 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:00.276863 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:00.277309 (+ 446us) client_negotiation.cc:769] Sending connection context
0927 22:58:00.277631 (+ 322us) client_negotiation.cc:240] Negotiation successful
0927 22:58:00.278270 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":435,"threads_started":1}
I20260927 22:58:00.281752 10763 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34435
I20260927 22:58:00.282258 10763 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:00.283432 10763 heartbeater.cc:511] Master 127.4.74.190:34435 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 22:58:00.286235 10574 ts_manager.cc:194] Registered new tserver with Master: 777d65cebeda4de0baf50caf516ee3f9 (127.4.74.129:39233)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.130@KRBTEST.COM" created.
I20260927 22:58:00.288117 10574 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:45275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab.
Entry for principal oryx/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab.
I20260927 22:58:00.320459 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34435
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab
--principal=oryx/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:00.744688 10769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:00.745093 10769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:00.745204 10769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:00.753630 10769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:00.753793 10769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:00.753929 10769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:58:00.766345 10769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab
--principal=oryx/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:00.768232 10769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:00.769939 10769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:00.787025 10774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:00.787087 10775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:00.787851 10777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:00.788424 10769 server_base.cc:1034] running on GCE node
I20260927 22:58:00.789224 10769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:00.790349 10769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:00.791654 10769 hybrid_clock.cc:648] HybridClock initialized: now 1790549880791586 us; error 51 us; skew 500 ppm
I20260927 22:58:00.792052 10769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:00 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549880, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:00.795591 10769 init.cc:377] Logged in from keytab as oryx/127.4.74.130@KRBTEST.COM (short username oryx)
I20260927 22:58:00.797302 10769 webserver.cc:460] Webserver started at http://127.4.74.130:38807/ using document root <none> and password file <none>
I20260927 22:58:00.798076 10769 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:00.798194 10769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:00.798581 10769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:00.801782 10769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8"
format_stamp: "Formatted at 2026-09-27 22:58:00 on dist-test-slave-5pd3"
server_key: "9bb2f3bb6b10f8df9b5f7bba6f67f839"
server_key_iv: "8cc41a67a680a81095e9941faaee696e"
server_key_version: "encryptionkey@0"
I20260927 22:58:00.802551 10769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8"
format_stamp: "Formatted at 2026-09-27 22:58:00 on dist-test-slave-5pd3"
server_key: "9bb2f3bb6b10f8df9b5f7bba6f67f839"
server_key_iv: "8cc41a67a680a81095e9941faaee696e"
server_key_version: "encryptionkey@0"
I20260927 22:58:00.808713 10769 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260927 22:58:00.812899 10784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:00.814709 10769 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 22:58:00.814888 10769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8"
format_stamp: "Formatted at 2026-09-27 22:58:00 on dist-test-slave-5pd3"
server_key: "9bb2f3bb6b10f8df9b5f7bba6f67f839"
server_key_iv: "8cc41a67a680a81095e9941faaee696e"
server_key_version: "encryptionkey@0"
I20260927 22:58:00.815081 10769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:00.839886 10769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:00.844715 10769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:00.845003 10769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:00.846052 10769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:00.847702 10769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:00.847785 10769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:00.847877 10769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:00.847929 10769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:00.888923 10769 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:45903
I20260927 22:58:00.889037 10896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:45903 every 8 connection(s)
I20260927 22:58:00.890700 10769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:58:00.901043 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 10769
I20260927 22:58:00.901242 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:58:00.901820 4394 external_mini_cluster.cc:1428] Setting key b198d991413ad2f5b1755190454dd213
Sep 27 22:58:00 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549880, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:00.918212 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:00.893882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:48965 (local address 127.4.74.190:34435)
0927 22:58:00.894468 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:00.894475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:00.895678 (+ 1203us) server_negotiation.cc:407] Connection header received
0927 22:58:00.898217 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:00.898223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:00.898333 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:00.898493 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:00.900966 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:00.901958 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:00.904389 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:00.905030 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:00.905193 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:00.909488 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:00.909499 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:00.909508 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:00.909575 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:00.913507 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:00.914780 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:00.914785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:00.914788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:00.915056 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:00.915735 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:00.915740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:00.915744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:00.916097 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:00.916459 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:00.917490 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0927 22:58:00.917780 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":276,"threads_started":1}
I20260927 22:58:00.919870 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:00.894378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.130:48965)
0927 22:58:00.895360 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:00.895416 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:00.897756 (+ 2340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:00.898885 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:00.898897 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:00.899698 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:00.900615 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:00.900651 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:00.902178 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:00.902184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:00.904161 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:00.904178 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:00.905357 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:00.905365 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:00.905660 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:00.906505 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:00.906530 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:00.909071 (+ 2541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:00.913803 (+ 4732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:00.913818 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:00.913843 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:00.914580 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:00.915234 (+ 654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:00.915240 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:00.915244 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:00.915539 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:00.916439 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:00.916450 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:00.917018 (+ 568us) client_negotiation.cc:769] Sending connection context
0927 22:58:00.917367 (+ 349us) client_negotiation.cc:240] Negotiation successful
0927 22:58:00.918124 (+ 757us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":370,"threads_started":1}
I20260927 22:58:00.921958 10897 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34435
I20260927 22:58:00.922358 10897 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:00.923374 10897 heartbeater.cc:511] Master 127.4.74.190:34435 requested a full tablet report, sending...
I20260927 22:58:00.925631 10574 ts_manager.cc:194] Registered new tserver with Master: dd54ebab9a1a4ce09fcf9ed9c02012b8 (127.4.74.130:45903)
I20260927 22:58:00.926702 10574 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:48965
WARNING: no policy specified for oryx/127.4.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab.
Entry for principal oryx/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab.
I20260927 22:58:00.961373 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34435
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab
--principal=oryx/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:58:01.291357 10763 heartbeater.cc:503] Master 127.4.74.190:34435 was elected leader, sending a full tablet report...
W20260927 22:58:01.327992 10903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:01.328303 10903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:01.328410 10903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:01.335726 10903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:01.335860 10903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:01.335979 10903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:58:01.345958 10903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab
--principal=oryx/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:01.347291 10903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:01.348726 10903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:01.363178 10911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:01.363278 10908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:01.363204 10909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:01.364380 10903 server_base.cc:1034] running on GCE node
I20260927 22:58:01.365183 10903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:01.366427 10903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:01.367682 10903 hybrid_clock.cc:648] HybridClock initialized: now 1790549881367608 us; error 81 us; skew 500 ppm
I20260927 22:58:01.368084 10903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:01 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549881, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:01.371997 10903 init.cc:377] Logged in from keytab as oryx/127.4.74.131@KRBTEST.COM (short username oryx)
I20260927 22:58:01.374006 10903 webserver.cc:460] Webserver started at http://127.4.74.131:37869/ using document root <none> and password file <none>
I20260927 22:58:01.375062 10903 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:01.375221 10903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:01.375684 10903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:01.379431 10903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "4b0cc13855954abc84ce0b436c7eed06"
format_stamp: "Formatted at 2026-09-27 22:58:01 on dist-test-slave-5pd3"
server_key: "6a6bd86ad1d3685ee1a1f8ca9677e624"
server_key_iv: "73ef82fca6a85350fd28ac51e09d5893"
server_key_version: "encryptionkey@0"
I20260927 22:58:01.380440 10903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "4b0cc13855954abc84ce0b436c7eed06"
format_stamp: "Formatted at 2026-09-27 22:58:01 on dist-test-slave-5pd3"
server_key: "6a6bd86ad1d3685ee1a1f8ca9677e624"
server_key_iv: "73ef82fca6a85350fd28ac51e09d5893"
server_key_version: "encryptionkey@0"
I20260927 22:58:01.387305 10903 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260927 22:58:01.391942 10918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:01.393924 10903 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:58:01.394212 10903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "4b0cc13855954abc84ce0b436c7eed06"
format_stamp: "Formatted at 2026-09-27 22:58:01 on dist-test-slave-5pd3"
server_key: "6a6bd86ad1d3685ee1a1f8ca9677e624"
server_key_iv: "73ef82fca6a85350fd28ac51e09d5893"
server_key_version: "encryptionkey@0"
I20260927 22:58:01.394569 10903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:01.428139 10903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:01.433357 10903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:01.433712 10903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:01.435050 10903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:01.436830 10903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:01.436913 10903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:01.437011 10903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:01.437068 10903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:01.478139 10903 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:39947
I20260927 22:58:01.478230 11030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:39947 every 8 connection(s)
I20260927 22:58:01.479908 10903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:58:01.486619 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 10903
I20260927 22:58:01.486842 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:58:01.487473 4394 external_mini_cluster.cc:1428] Setting key 4041f240fbf94274cb8bd2e0bc5dcc0e
Sep 27 22:58:01 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549881, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:01.505950 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:01.483112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:33929 (local address 127.4.74.190:34435)
0927 22:58:01.483740 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:01.483748 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:01.485112 (+ 1364us) server_negotiation.cc:407] Connection header received
0927 22:58:01.487667 (+ 2555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:01.487672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:01.487779 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:01.487911 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:01.490179 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:01.491383 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:01.493526 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:01.494005 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:01.494128 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:01.498256 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:01.498261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:01.498265 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:01.498312 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:01.501597 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:01.502825 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:01.502831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:01.502836 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:01.503011 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:01.503728 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:01.503736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:01.503742 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:01.504155 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:01.504324 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:01.505400 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0927 22:58:01.505621 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":291,"threads_started":1}
I20260927 22:58:01.506948 11034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:01.483962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.131:33929)
0927 22:58:01.484876 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:01.484927 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:01.487213 (+ 2286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:01.488232 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:01.488242 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:01.488990 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:01.489972 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:01.490000 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:01.491627 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:01.491631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:01.493318 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:01.493335 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:01.494305 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:01.494311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:01.494556 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:01.495330 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:01.495356 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:01.497923 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:01.501854 (+ 3931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:01.501865 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:01.501885 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:01.502630 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:01.503228 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:01.503235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:01.503239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:01.503521 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:01.504316 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:01.504323 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:01.504908 (+ 585us) client_negotiation.cc:769] Sending connection context
0927 22:58:01.505226 (+ 318us) client_negotiation.cc:240] Negotiation successful
0927 22:58:01.505653 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":372,"threads_started":1}
I20260927 22:58:01.508896 11031 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34435
I20260927 22:58:01.509347 11031 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:01.510330 11031 heartbeater.cc:511] Master 127.4.74.190:34435 requested a full tablet report, sending...
I20260927 22:58:01.512173 10574 ts_manager.cc:194] Registered new tserver with Master: 4b0cc13855954abc84ce0b436c7eed06 (127.4.74.131:39947)
I20260927 22:58:01.513163 10574 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:33929
I20260927 22:58:01.517246 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.74.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.189.keytab.
Entry for principal oryx/127.4.74.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.189.keytab.
I20260927 22:58:01.580354 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.189:40471
--webserver_interface=127.4.74.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.189.keytab
--principal=oryx/127.4.74.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:34435,127.4.74.189:40471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:58:01.930573 10897 heartbeater.cc:503] Master 127.4.74.190:34435 was elected leader, sending a full tablet report...
W20260927 22:58:02.001124 11037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:02.001596 11037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:02.001777 11037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:02.010255 11037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:58:02.010526 11037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:02.010641 11037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:02.010748 11037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:58:02.010839 11037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:58:02.024272 11037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:34435,127.4.74.189:40471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.189.keytab
--principal=oryx/127.4.74.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.189:40471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.74.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
W20260927 22:58:02.024845 11037 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.74.190:34435,127.4.74.189:40471'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260927 22:58:02.026286 11037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:02.028219 11037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:02.043099 11045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:02.043315 11042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:02.043237 11043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:02.044883 11037 server_base.cc:1034] running on GCE node
I20260927 22:58:02.045940 11037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:02.047816 11037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:02.049057 11037 hybrid_clock.cc:648] HybridClock initialized: now 1790549882048953 us; error 83 us; skew 500 ppm
I20260927 22:58:02.049486 11037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549882, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:02.053282 11037 init.cc:377] Logged in from keytab as oryx/127.4.74.189@KRBTEST.COM (short username oryx)
I20260927 22:58:02.055313 11037 webserver.cc:460] Webserver started at http://127.4.74.189:38639/ using document root <none> and password file <none>
I20260927 22:58:02.056277 11037 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:02.056416 11037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:02.056795 11037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:02.060395 11037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data/instance:
uuid: "7e424f35d1d146b58d3b60a554e7d0c4"
format_stamp: "Formatted at 2026-09-27 22:58:02 on dist-test-slave-5pd3"
server_key: "933facf8d2332c5bd8c6c0d14392fdf5"
server_key_iv: "91e1defe1d878b7ed2a281d19bd55550"
server_key_version: "encryptionkey@0"
I20260927 22:58:02.061280 11037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal/instance:
uuid: "7e424f35d1d146b58d3b60a554e7d0c4"
format_stamp: "Formatted at 2026-09-27 22:58:02 on dist-test-slave-5pd3"
server_key: "933facf8d2332c5bd8c6c0d14392fdf5"
server_key_iv: "91e1defe1d878b7ed2a281d19bd55550"
server_key_version: "encryptionkey@0"
I20260927 22:58:02.068004 11037 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 22:58:02.072392 11052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:02.074419 11037 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:58:02.074676 11037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal
uuid: "7e424f35d1d146b58d3b60a554e7d0c4"
format_stamp: "Formatted at 2026-09-27 22:58:02 on dist-test-slave-5pd3"
server_key: "933facf8d2332c5bd8c6c0d14392fdf5"
server_key_iv: "91e1defe1d878b7ed2a281d19bd55550"
server_key_version: "encryptionkey@0"
I20260927 22:58:02.075030 11037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:02.099929 11037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:02.106613 11037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:02.106951 11037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:02.129513 11037 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.189:40471
I20260927 22:58:02.129547 11103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.189:40471 every 8 connection(s)
I20260927 22:58:02.131068 11037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
I20260927 22:58:02.135948 11104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:02.138610 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 11037
I20260927 22:58:02.146080 11104 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.189@KRBTEST.COM
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549882, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.189@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:02.168224 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.141535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52792 (local address 127.4.74.189:40471)
0927 22:58:02.142735 (+ 1200us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.142760 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.142825 (+ 65us) server_negotiation.cc:407] Connection header received
0927 22:58:02.145384 (+ 2559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.145420 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.146080 (+ 660us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.146658 (+ 578us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.148730 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.150592 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.152323 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.153635 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.153773 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.158956 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.158968 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.158986 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.159027 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.162516 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.163432 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.163443 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.163457 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.163735 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.164262 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.164266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.164268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.164920 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.165186 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.165973 (+ 787us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.166513 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":874,"thread_start_us":492,"threads_started":1}
I20260927 22:58:02.168759 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.149980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45954 (local address 127.4.74.190:34435)
0927 22:58:02.151505 (+ 1525us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.151510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.151526 (+ 16us) server_negotiation.cc:407] Connection header received
0927 22:58:02.151648 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.151651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.151733 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.151861 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.153583 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.154721 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.156236 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.156914 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.157055 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.161568 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.161573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.161577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.161651 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.165259 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.166297 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.166302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.166304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.166457 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.166911 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.166916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.166918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.167269 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.167407 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.168095 (+ 688us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.168352 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1412,"thread_start_us":1301,"threads_started":1}
I20260927 22:58:02.168766 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.149898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:45954)
0927 22:58:02.150816 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:02.150874 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:02.151065 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:02.151940 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:02.151948 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:02.152930 (+ 982us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:02.153379 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.153400 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.154890 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.154893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.156080 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.156092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.157200 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.157209 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.157400 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.158189 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:02.158215 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:02.161325 (+ 3110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:02.165481 (+ 4156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.165491 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.165506 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.166172 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.166602 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.166607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.166611 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.166784 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.167424 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:02.167431 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:02.167688 (+ 257us) client_negotiation.cc:769] Sending connection context
0927 22:58:02.167930 (+ 242us) client_negotiation.cc:240] Negotiation successful
0927 22:58:02.168322 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":1,"thread_start_us":419,"threads_started":1}
I20260927 22:58:02.171919 11104 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549882, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.189@KRBTEST.COM for oryx/127.4.74.189@KRBTEST.COM
I20260927 22:58:02.186973 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.172713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:40471 (local address 127.0.0.1:52798)
0927 22:58:02.173062 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:02.173082 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:02.173195 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:02.173553 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:02.173556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:02.174095 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:02.174346 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.174357 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.175539 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.175600 (+ 61us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.176596 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.176609 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.177414 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.177419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.177589 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.178319 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:02.178338 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:02.180715 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:02.184270 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.184276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.184279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.184796 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.185246 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.185249 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.185252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.185400 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.185997 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:02.186001 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:02.186185 (+ 184us) client_negotiation.cc:769] Sending connection context
0927 22:58:02.186326 (+ 141us) client_negotiation.cc:240] Negotiation successful
0927 22:58:02.186597 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260927 22:58:02.187131 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.173024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52798 (local address 127.4.74.189:40471)
0927 22:58:02.173195 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.173200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.173227 (+ 27us) server_negotiation.cc:407] Connection header received
0927 22:58:02.173282 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.173285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.173349 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.173439 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.174487 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.175373 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.176745 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.177244 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.177361 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.180914 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.180920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.180924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.180975 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.184065 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.184963 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.184968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.184970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.185119 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.185532 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.185536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.185538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.185855 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.185977 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.186430 (+ 453us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.186748 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":59}
I20260927 22:58:02.192786 11104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: Bootstrap starting.
I20260927 22:58:02.196857 11104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:02.198315 11104 log.cc:824] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:02.202414 11104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: No bootstrap required, opened a new log
I20260927 22:58:02.211251 11104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } }
I20260927 22:58:02.211840 11104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:02.211958 11104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e424f35d1d146b58d3b60a554e7d0c4, State: Initialized, Role: FOLLOWER
I20260927 22:58:02.212743 11104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } }
I20260927 22:58:02.213939 11111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } } }
I20260927 22:58:02.214426 11111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 22:58:02.215164 11104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 22:58:02.226775 11122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 22:58:02.226953 11122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 22:58:02.233669 11037 master_runner.cc:428] Detected that this master 7e424f35d1d146b58d3b60a554e7d0c4 is joining an existing cluster
I20260927 22:58:02.233772 11037 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.74.189:40471
WARNING: no policy specified for oryx/127.4.74.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.188@KRBTEST.COM" created.
I20260927 22:58:02.265401 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.244845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.4.74.190:34435)
0927 22:58:02.245086 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.245094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.245840 (+ 746us) server_negotiation.cc:407] Connection header received
0927 22:58:02.246164 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.246173 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.246522 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.246622 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.248371 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.249538 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.251860 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.252894 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.253126 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.257564 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.257573 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.257581 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.257630 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.261611 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.262582 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.262590 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.262597 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.262764 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.263413 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.263421 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.263427 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.263886 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.264073 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.264687 (+ 614us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.264952 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260927 22:58:02.266031 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.244801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:45960)
0927 22:58:02.245688 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:02.245713 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:02.245985 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:02.246867 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:02.246872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:02.247764 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:02.248172 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.248186 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.249772 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.249777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.251599 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.251615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.253188 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.253196 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.253618 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.254594 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:02.254639 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:02.257312 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:02.261810 (+ 4498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.261818 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.261825 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.262427 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.262897 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.262904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.262910 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.263269 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.264037 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:02.264049 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:02.264292 (+ 243us) client_negotiation.cc:769] Sending connection context
0927 22:58:02.264530 (+ 238us) client_negotiation.cc:240] Negotiation successful
0927 22:58:02.264931 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":486,"threads_started":1}
I20260927 22:58:02.275992 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.246457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52800 (local address 127.4.74.189:40471)
0927 22:58:02.246681 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.246689 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.256863 (+ 10174us) server_negotiation.cc:407] Connection header received
0927 22:58:02.257072 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.257080 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.257163 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.257242 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.258783 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.259980 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.262048 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.262829 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.262938 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.267306 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.267315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.267323 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.267363 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.271596 (+ 4233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.273259 (+ 1663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.273268 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.273275 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.273455 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.273934 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.273941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.273948 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.274441 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.274588 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.275138 (+ 550us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.275444 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 22:58:02.277513 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.245953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:40471 (local address 127.0.0.1:52800)
0927 22:58:02.256697 (+ 10744us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:02.256721 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:02.256910 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:02.257495 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:02.257500 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:02.258195 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:02.258601 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.258615 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.260205 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.260211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.261889 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.261904 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.263360 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.263367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.263573 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.265610 (+ 2037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:02.265651 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:02.267103 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:02.271809 (+ 4706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.271817 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.271824 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.272408 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.273603 (+ 1195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.273611 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.273618 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.273816 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.274688 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:02.274695 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:02.274860 (+ 165us) client_negotiation.cc:769] Sending connection context
0927 22:58:02.276420 (+ 1560us) client_negotiation.cc:240] Negotiation successful
0927 22:58:02.276827 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10566,"thread_start_us":10173,"threads_started":1}
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549879, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.190@KRBTEST.COM for oryx/127.4.74.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.188.keytab.
Entry for principal oryx/127.4.74.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.188.keytab.
I20260927 22:58:02.304134 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.188:34933
--webserver_interface=127.4.74.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.188.keytab
--principal=oryx/127.4.74.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:34435,127.4.74.189:40471,127.4.74.188:34933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:58:02.309854 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.275866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.4.74.189:40471)
0927 22:58:02.276370 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.276378 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.278891 (+ 2513us) server_negotiation.cc:407] Connection header received
0927 22:58:02.279416 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.279424 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.279611 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.279710 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.281777 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.283165 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.284805 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.285860 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.286065 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.291246 (+ 5181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.291258 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.291264 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.291387 (+ 123us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.305737 (+ 14350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.306700 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.306707 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.306711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.306867 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.307271 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.307277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.307280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.307711 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.308011 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.308833 (+ 822us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.309194 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":2069,"threads_started":1}
I20260927 22:58:02.309908 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.275301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:40471 (local address 127.0.0.1:52802)
0927 22:58:02.278698 (+ 3397us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:02.278764 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:02.279074 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:02.280117 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:02.280127 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:02.281290 (+ 1163us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:02.281572 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.281582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.283447 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.283452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.284611 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.284626 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.286118 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.286124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.286316 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.287372 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:02.287391 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:02.290823 (+ 3432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:02.305984 (+ 15161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.306006 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.306015 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.306551 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.306991 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.306998 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.307001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.307152 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.307977 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:02.307991 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:02.308208 (+ 217us) client_negotiation.cc:769] Sending connection context
0927 22:58:02.308660 (+ 452us) client_negotiation.cc:240] Negotiation successful
0927 22:58:02.309186 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3051,"thread_start_us":5062,"threads_started":1}
I20260927 22:58:02.314594 10574 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.74.189:40471: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4"
member_type: NON_VOTER
last_known_addr {
host: "127.4.74.189"
port: 40471
}
attrs {
promote: true
}
}
dest_uuid: "f5325a6273cb4a218f13ac8cb7dd5537"
cas_config_opid_index: -1
I20260927 22:58:02.316468 10574 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } }
I20260927 22:58:02.332968 11079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:02.340586 11079 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 FOLLOWER]: Refusing update from remote peer f5325a6273cb4a218f13ac8cb7dd5537: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260927 22:58:02.352032 11138 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:02.351373 11137 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7e424f35d1d146b58d3b60a554e7d0c4 (127.4.74.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:02.357578 11137 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.74.189:40471
I20260927 22:58:02.358714 11037 master.cc:537] Master@127.4.74.189:40471 shutting down...
W20260927 22:58:02.360013 11058 tablet.cc:2369] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:58:02.360759 11138 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:02.361423 11138 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:02.398512 11079 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7e424f35d1d146b58d3b60a554e7d0c4 (127.4.74.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:02.414659 11111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5325a6273cb4a218f13ac8cb7dd5537. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:02.415056 11111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: LEARNER
I20260927 22:58:02.415464 11142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:02.415786 11142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: LEARNER
I20260927 22:58:02.416086 11146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:02.416426 11146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: LEARNER
I20260927 22:58:02.417050 11037 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 LEARNER]: Raft consensus shutting down.
I20260927 22:58:02.417652 11037 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 LEARNER]: Raft consensus is shut down!
I20260927 22:58:02.417855 11037 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: stopping tablet replica
I20260927 22:58:02.437850 11037 master.cc:552] Master@127.4.74.189:40471 shutdown complete.
I20260927 22:58:02.437963 11037 master_runner.cc:305] Clearing existing system tablet
I20260927 22:58:02.438421 11037 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:02.438530 11037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:02.442111 11037 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260927 22:58:02.446092 11148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:02.446647 11037 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:02.447048 11037 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260927 22:58:02.447175 11037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal
uuid: "7e424f35d1d146b58d3b60a554e7d0c4"
format_stamp: "Formatted at 2026-09-27 22:58:02 on dist-test-slave-5pd3"
server_key: "933facf8d2332c5bd8c6c0d14392fdf5"
server_key_iv: "91e1defe1d878b7ed2a281d19bd55550"
server_key_version: "encryptionkey@0"
I20260927 22:58:02.452284 11037 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:02.460204 11037 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 22:58:02.460446 11037 log.cc:1196] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260927 22:58:02.460943 11037 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: Deleting consensus metadata
I20260927 22:58:02.461244 11037 master_runner.cc:315] Copying system tablet from 127.4.74.190:34435
I20260927 22:58:02.464339 11037 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: tablet copy: Beginning tablet copy session from remote peer at address 127.4.74.190:34435
I20260927 22:58:02.483681 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.465399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:45970)
0927 22:58:02.466227 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:02.466255 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:02.466498 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:02.467083 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:02.467088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:02.467701 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:02.468032 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.468044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.469732 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.469738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.471311 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.471331 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.472502 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.472509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.472715 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.473820 (+ 1105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:02.473847 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:02.475044 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:02.480256 (+ 5212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.480263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.480267 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.480875 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.481373 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.481378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.481382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.481549 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.482334 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:02.482340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:02.482535 (+ 195us) client_negotiation.cc:769] Sending connection context
0927 22:58:02.482702 (+ 167us) client_negotiation.cc:240] Negotiation successful
0927 22:58:02.483123 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":442,"threads_started":1}
I20260927 22:58:02.484946 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.465380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45970 (local address 127.4.74.190:34435)
0927 22:58:02.465625 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.465634 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.466430 (+ 796us) server_negotiation.cc:407] Connection header received
0927 22:58:02.466744 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.466751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.466868 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.467007 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.468233 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.469483 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.471498 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.472304 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.472456 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.475313 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.475320 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.475327 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.475407 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.480002 (+ 4595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.481052 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.481058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.481062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.481230 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.481680 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.481685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.481688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.482115 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.482266 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.484278 (+ 2012us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.484553 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260927 22:58:02.486224 10594 tablet_copy_service.cc:140] P f5325a6273cb4a218f13ac8cb7dd5537: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7e424f35d1d146b58d3b60a554e7d0c4 ({username='oryx', principal='oryx/127.4.74.189@KRBTEST.COM'} at 127.0.0.1:45970)
I20260927 22:58:02.486526 10594 tablet_copy_service.cc:161] P f5325a6273cb4a218f13ac8cb7dd5537: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7e424f35d1d146b58d3b60a554e7d0c4 at {username='oryx', principal='oryx/127.4.74.189@KRBTEST.COM'} at 127.0.0.1:45970: session id = 7e424f35d1d146b58d3b60a554e7d0c4-00000000000000000000000000000000
I20260927 22:58:02.490960 10594 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 22:58:02.493606 11037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 22:58:02.500497 11037 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: tablet copy: Starting download of 0 data blocks...
I20260927 22:58:02.500837 11037 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: tablet copy: Starting download of 1 WAL segments...
I20260927 22:58:02.504295 11037 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 22:58:02.509677 10594 tablet_copy_service.cc:342] P f5325a6273cb4a218f13ac8cb7dd5537: Request end of tablet copy session 7e424f35d1d146b58d3b60a554e7d0c4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.74.189@KRBTEST.COM'} at 127.0.0.1:45970
I20260927 22:58:02.509934 10594 tablet_copy_service.cc:433] P f5325a6273cb4a218f13ac8cb7dd5537: ending tablet copy session 7e424f35d1d146b58d3b60a554e7d0c4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7e424f35d1d146b58d3b60a554e7d0c4
I20260927 22:58:02.516707 11031 heartbeater.cc:503] Master 127.4.74.190:34435 was elected leader, sending a full tablet report...
I20260927 22:58:02.535789 11037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:02.548453 11155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:02.549391 11156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:02.560138 11158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:02.560870 11037 server_base.cc:1034] running on GCE node
I20260927 22:58:02.561517 11037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:02.562543 11037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:02.564041 11037 hybrid_clock.cc:648] HybridClock initialized: now 1790549882563940 us; error 61 us; skew 500 ppm
I20260927 22:58:02.564471 11037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549882, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:02.567225 11037 init.cc:377] Logged in from keytab as oryx/127.4.74.189@KRBTEST.COM (short username oryx)
I20260927 22:58:02.572682 11037 webserver.cc:460] Webserver started at http://127.4.74.189:36983/ using document root <none> and password file <none>
I20260927 22:58:02.573171 11037 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:02.573323 11037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:02.577028 11037 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260927 22:58:02.579418 11165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:02.579921 11037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260927 22:58:02.580132 11037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal
uuid: "7e424f35d1d146b58d3b60a554e7d0c4"
format_stamp: "Formatted at 2026-09-27 22:58:02 on dist-test-slave-5pd3"
server_key: "933facf8d2332c5bd8c6c0d14392fdf5"
server_key_iv: "91e1defe1d878b7ed2a281d19bd55550"
server_key_version: "encryptionkey@0"
I20260927 22:58:02.580384 11037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:02.617252 11037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:02.622339 11037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:02.664961 11037 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.189:40471
I20260927 22:58:02.664996 11216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.189:40471 every 8 connection(s)
I20260927 22:58:02.666229 11037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
I20260927 22:58:02.673458 11217 sys_catalog.cc:263] Verifying existing consensus state
I20260927 22:58:02.679102 11217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: Bootstrap starting.
I20260927 22:58:02.696820 11217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 22:58:02.697485 11217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: Bootstrap complete.
I20260927 22:58:02.698814 11217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } }
I20260927 22:58:02.699205 11217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e424f35d1d146b58d3b60a554e7d0c4, State: Initialized, Role: LEARNER
I20260927 22:58:02.699570 11217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } }
I20260927 22:58:02.701079 11217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:58:02.700817 11220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:02.701754 11220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: LEARNER
I20260927 22:58:02.710594 11231 catalog_manager.cc:1259] Loaded cluster ID: 24c7cb2cc7914dd0be03d66f9624828c
I20260927 22:58:02.710786 11231 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: loading cluster ID for follower catalog manager: success
I20260927 22:58:02.715060 11231 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: acquiring CA information for follower catalog manager: success
I20260927 22:58:02.718628 11231 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 22:58:02.785725 11134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:02.786103 11134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:02.786267 11134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:02.793995 11134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:58:02.794170 11134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:02.794282 11134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:02.794378 11134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:58:02.794466 11134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:58:02.805401 11134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:34435,127.4.74.189:40471,127.4.74.188:34933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.188.keytab
--principal=oryx/127.4.74.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.188:34933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.74.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:02.806819 11134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:02.808229 11134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:02.822554 11236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:02.822666 11239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:02.822662 11237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:02.823625 11134 server_base.cc:1034] running on GCE node
I20260927 22:58:02.824616 11134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:02.826251 11134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:02.827517 11134 hybrid_clock.cc:648] HybridClock initialized: now 1790549882827454 us; error 56 us; skew 500 ppm
I20260927 22:58:02.828013 11134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549882, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:02.831537 11134 init.cc:377] Logged in from keytab as oryx/127.4.74.188@KRBTEST.COM (short username oryx)
I20260927 22:58:02.833348 11134 webserver.cc:460] Webserver started at http://127.4.74.188:38895/ using document root <none> and password file <none>
I20260927 22:58:02.834115 11134 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:02.834277 11134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:02.834595 11134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:02.837869 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data/instance:
uuid: "812db8d443af4789957296a6b7a6dc53"
format_stamp: "Formatted at 2026-09-27 22:58:02 on dist-test-slave-5pd3"
server_key: "96b638d1dfc0aefa3dabf47773f33fe4"
server_key_iv: "e4bedeb185f59ddf401a62ff4deb8c0c"
server_key_version: "encryptionkey@0"
I20260927 22:58:02.838766 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal/instance:
uuid: "812db8d443af4789957296a6b7a6dc53"
format_stamp: "Formatted at 2026-09-27 22:58:02 on dist-test-slave-5pd3"
server_key: "96b638d1dfc0aefa3dabf47773f33fe4"
server_key_iv: "e4bedeb185f59ddf401a62ff4deb8c0c"
server_key_version: "encryptionkey@0"
I20260927 22:58:02.845208 11134 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260927 22:58:02.849363 11246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:02.851188 11134 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260927 22:58:02.851434 11134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal
uuid: "812db8d443af4789957296a6b7a6dc53"
format_stamp: "Formatted at 2026-09-27 22:58:02 on dist-test-slave-5pd3"
server_key: "96b638d1dfc0aefa3dabf47773f33fe4"
server_key_iv: "e4bedeb185f59ddf401a62ff4deb8c0c"
server_key_version: "encryptionkey@0"
I20260927 22:58:02.851727 11134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:02.879029 11134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:02.883621 11134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:02.883920 11134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:02.907718 11134 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.188:34933
I20260927 22:58:02.907752 11297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.188:34933 every 8 connection(s)
I20260927 22:58:02.909338 11134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
I20260927 22:58:02.910239 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 11134
I20260927 22:58:02.915011 11298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:02.923393 11298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.188@KRBTEST.COM
I20260927 22:58:02.934952 11303 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:02.935325 11303 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549882, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.188@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:02.943679 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.913507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.4.74.188:34933)
0927 22:58:02.914564 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.914585 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.914639 (+ 54us) server_negotiation.cc:407] Connection header received
0927 22:58:02.916996 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.917027 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.917550 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.918065 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.920472 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.922162 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.923741 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.924933 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.925165 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.932010 (+ 6845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.932025 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.932041 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.932102 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.937118 (+ 5016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.938285 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.938301 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.938313 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.938600 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.939092 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.939100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.939108 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.939987 (+ 879us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.940362 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.940906 (+ 544us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.941595 (+ 689us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":455,"threads_started":1}
I20260927 22:58:02.946274 11307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.936626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52812 (local address 127.4.74.189:40471)
0927 22:58:02.937738 (+ 1112us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.937746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.937765 (+ 19us) server_negotiation.cc:407] Connection header received
0927 22:58:02.937835 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.937840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.937947 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.938077 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:02.938726 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.940257 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.942869 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.944717 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.944974 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.945284 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.945729 (+ 445us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.945876 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":982,"thread_start_us":789,"threads_started":1}
I20260927 22:58:02.946609 11306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.936053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:40471 (local address 127.0.0.1:52812)
0927 22:58:02.937040 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:02.937063 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:02.937201 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:02.938181 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:02.938189 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:02.938233 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:02.938539 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.938552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.940416 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.940421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.942621 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.942647 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.945021 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.945032 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.945275 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.945354 (+ 79us) client_negotiation.cc:769] Sending connection context
0927 22:58:02.945454 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 22:58:02.946277 (+ 823us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":770,"threads_started":1}
I20260927 22:58:02.947664 11302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.926471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:45976)
0927 22:58:02.927270 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:02.927322 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:02.927503 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:02.928309 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:02.928315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:02.928983 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:02.929275 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.929284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.930682 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.930689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.932354 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.932370 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.934287 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.934298 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.934590 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.936891 (+ 2301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:02.936926 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:02.940099 (+ 3173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:02.944184 (+ 4085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.944198 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.944208 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.944726 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.945232 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.945237 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.945240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.945399 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.946152 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:02.946161 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:02.946451 (+ 290us) client_negotiation.cc:769] Sending connection context
0927 22:58:02.946708 (+ 257us) client_negotiation.cc:240] Negotiation successful
0927 22:58:02.947159 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":366,"threads_started":1}
I20260927 22:58:02.947983 11220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5325a6273cb4a218f13ac8cb7dd5537. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:02.948328 11220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: LEARNER
I20260927 22:58:02.948352 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.926758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45976 (local address 127.4.74.190:34435)
0927 22:58:02.927034 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.927041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.927738 (+ 697us) server_negotiation.cc:407] Connection header received
0927 22:58:02.927846 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.927853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.927985 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.928218 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.929489 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.930453 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.932535 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.933279 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.935038 (+ 1759us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.940373 (+ 5335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.940381 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.940386 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.940434 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.944017 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.944865 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.944871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.944875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.945076 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.945576 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.945581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.945584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.945959 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.946127 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.947290 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.947637 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260927 22:58:02.949021 11304 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537: attempting to promote NON_VOTER 7e424f35d1d146b58d3b60a554e7d0c4 to VOTER
I20260927 22:58:02.950150 11298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } has no permanent_uuid. Determining permanent_uuid...
I20260927 22:58:02.950098 11304 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } }
I20260927 22:58:02.952454 11192 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 LEARNER]: Refusing update from remote peer f5325a6273cb4a218f13ac8cb7dd5537: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 22:58:02.953404 11304 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260927 22:58:02.958056 11303 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 7e424f35d1d146b58d3b60a554e7d0c4 (127.4.74.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } }
I20260927 22:58:02.959108 11192 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 7e424f35d1d146b58d3b60a554e7d0c4 (127.4.74.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } }
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549882, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.188@KRBTEST.COM for oryx/127.4.74.189@KRBTEST.COM
I20260927 22:58:02.962074 11303 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } }
I20260927 22:58:02.962417 11303 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:02.962764 11303 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } }
I20260927 22:58:02.962972 11303 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:02.963862 11220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } }
I20260927 22:58:02.964167 11220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 22:58:02.968644 11307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.951248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52820 (local address 127.4.74.189:40471)
0927 22:58:02.951738 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.951744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.951766 (+ 22us) server_negotiation.cc:407] Connection header received
0927 22:58:02.951887 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.951891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.951970 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.952092 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.953057 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.954039 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.955593 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.956443 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.956571 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.961342 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.961350 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.961362 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.961415 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.965595 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.966571 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.966575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.966578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.966773 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.967203 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.967207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.967209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.967503 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.967761 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.968185 (+ 424us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.968390 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343}
I20260927 22:58:02.968659 11302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.950932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:40471 (local address 127.0.0.1:52820)
0927 22:58:02.951509 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:02.951536 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:02.951743 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:02.952179 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:02.952183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:02.952566 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:02.952835 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.952851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.954202 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.954206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.955393 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.955407 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.956641 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.956647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.956986 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.957854 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:02.957882 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:02.961036 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:02.965819 (+ 4783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.965825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.965828 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.966392 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.966897 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.966900 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.966903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.967049 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.967758 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:02.967763 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:02.967918 (+ 155us) client_negotiation.cc:769] Sending connection context
0927 22:58:02.968061 (+ 143us) client_negotiation.cc:240] Negotiation successful
0927 22:58:02.968359 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404}
I20260927 22:58:02.969990 11298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 22:58:02 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549882, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.188@KRBTEST.COM for oryx/127.4.74.188@KRBTEST.COM
I20260927 22:58:02.986537 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.970920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.4.74.188:34933)
0927 22:58:02.971148 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:02.971156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:02.971420 (+ 264us) server_negotiation.cc:407] Connection header received
0927 22:58:02.971620 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:02.971623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:02.971689 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:02.971777 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:02.972733 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.973943 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.975328 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.975882 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.975999 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.980175 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:02.980186 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:02.980196 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:02.980236 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:02.983267 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.984270 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.984275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.984278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.984444 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:02.984911 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:02.984915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:02.984917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:02.985225 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:02.985406 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:02.985792 (+ 386us) server_negotiation.cc:299] Negotiation successful
0927 22:58:02.986193 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":168,"server-negotiator.queue_time_us":75}
I20260927 22:58:02.986537 11302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:02.971043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:34933 (local address 127.0.0.1:43182)
0927 22:58:02.971319 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:02.971337 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:02.971463 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:02.971854 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:02.971858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:02.972308 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:02.972490 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.972499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.974213 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.974218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.975144 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:02.975156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:02.976091 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:02.976096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:02.976284 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:02.977135 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:02.977157 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:02.979899 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:02.983466 (+ 3567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.983471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.983475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.984088 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.984595 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:02.984599 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:02.984601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:02.984787 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:02.985365 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:02.985370 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:02.985546 (+ 176us) client_negotiation.cc:769] Sending connection context
0927 22:58:02.985703 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 22:58:02.986051 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260927 22:58:02.992867 11298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Bootstrap starting.
I20260927 22:58:02.997023 11298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:02.998361 11298 log.cc:824] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:03.002481 11298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: No bootstrap required, opened a new log
I20260927 22:58:03.011423 11298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } }
I20260927 22:58:03.012041 11298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:03.012209 11298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 812db8d443af4789957296a6b7a6dc53, State: Initialized, Role: FOLLOWER
I20260927 22:58:03.013201 11298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } }
I20260927 22:58:03.014350 11310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } } }
I20260927 22:58:03.014794 11310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 22:58:03.016258 11298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 22:58:03.027160 11321 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 22:58:03.027356 11321 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 22:58:03.033371 11134 master_runner.cc:428] Detected that this master 812db8d443af4789957296a6b7a6dc53 is joining an existing cluster
I20260927 22:58:03.033591 11134 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.74.188:34933
I20260927 22:58:03.052575 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 10635
I20260927 22:58:03.060200 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.039114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.4.74.190:34435)
0927 22:58:03.039326 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.039334 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.041270 (+ 1936us) server_negotiation.cc:407] Connection header received
0927 22:58:03.041513 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.041520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.041613 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.041691 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:03.043773 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.045016 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.047530 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.048265 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.048375 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.051798 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:03.051806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:03.051812 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:03.051861 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:03.056233 (+ 4372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.057402 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.057409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.057413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.057615 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.058112 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.058116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.058120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.058524 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:03.058686 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.059425 (+ 739us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.059761 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 22:58:03.061511 11327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.039161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:45984)
0927 22:58:03.041166 (+ 2005us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:03.041184 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:03.041358 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:03.041901 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:03.041911 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:03.042737 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:03.043536 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.043580 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.045250 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.045255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.047333 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.047347 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.048720 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.048726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.048935 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.049649 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:03.049677 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:03.051535 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:03.056561 (+ 5026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.056568 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.056573 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.057214 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.057780 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.057785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.057788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.057977 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.058768 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:03.058774 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:03.059044 (+ 270us) client_negotiation.cc:769] Sending connection context
0927 22:58:03.060548 (+ 1504us) client_negotiation.cc:240] Negotiation successful
0927 22:58:03.060945 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1817,"mutex_wait_us":260,"thread_start_us":606,"threads_started":1}
I20260927 22:58:03.062439 11307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.040976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.4.74.189:40471)
0927 22:58:03.041200 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.041207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.043251 (+ 2044us) server_negotiation.cc:407] Connection header received
0927 22:58:03.043464 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.043471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.043600 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.043680 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:03.045376 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.046475 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.048551 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.049251 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.049359 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.052607 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:03.052616 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:03.052620 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:03.053110 (+ 490us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:03.057772 (+ 4662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.058991 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.059000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.059007 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.059261 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.059864 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.059871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.059874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.060217 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:03.060738 (+ 521us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.061494 (+ 756us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.062034 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 22:58:03.068127 11329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.040550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:40471 (local address 127.0.0.1:52832)
0927 22:58:03.043089 (+ 2539us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:03.043111 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:03.043264 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:03.043910 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:03.043920 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:03.044834 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:03.045187 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.045195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.046648 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.046653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.048394 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.048406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.049725 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.049738 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.049959 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.050995 (+ 1036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:03.051022 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:03.052384 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:03.058086 (+ 5702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.058095 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.058102 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.058813 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.059445 (+ 632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.059452 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.059459 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.059706 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.060865 (+ 1159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:03.060872 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:03.061201 (+ 329us) client_negotiation.cc:769] Sending connection context
0927 22:58:03.066863 (+ 5662us) client_negotiation.cc:240] Negotiation successful
0927 22:58:03.067364 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2316,"thread_start_us":2066,"threads_started":1}
I20260927 22:58:03.069500 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.039657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.4.74.188:34933)
0927 22:58:03.039867 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.039876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.040364 (+ 488us) server_negotiation.cc:407] Connection header received
0927 22:58:03.040593 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.040600 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.040689 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.040810 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:03.043936 (+ 3126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.045163 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.047412 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.048139 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.048256 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.061316 (+ 13060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:03.061325 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:03.061330 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:03.061368 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:03.066062 (+ 4694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.066970 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.066976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.066980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.067125 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.067608 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.067612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.067614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.068104 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:03.068262 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.068780 (+ 518us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.069039 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 22:58:03.070597 11328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.039161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:34933 (local address 127.0.0.1:43188)
0927 22:58:03.040237 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:03.040260 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:03.040455 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:03.041237 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:03.041247 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:03.042979 (+ 1732us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:03.043784 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.043799 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.045408 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.045413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.047246 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.047267 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.048575 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.048581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.048799 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.049820 (+ 1021us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:03.049850 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:03.060460 (+ 10610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:03.066270 (+ 5810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.066275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.066280 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.066815 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.067261 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.067270 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.067277 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.067475 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.068330 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:03.068334 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:03.068521 (+ 187us) client_negotiation.cc:769] Sending connection context
0927 22:58:03.069755 (+ 1234us) client_negotiation.cc:240] Negotiation successful
0927 22:58:03.070112 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":82,"thread_start_us":1515,"threads_started":1}
Sep 27 22:58:03 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549879, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.190@KRBTEST.COM for oryx/127.4.74.188@KRBTEST.COM
Sep 27 22:58:03 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.129@KRBTEST.COM
I20260927 22:58:03.088629 11306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.070719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:34933 (local address 127.0.0.1:43196)
0927 22:58:03.072026 (+ 1307us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:03.072041 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:03.072197 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:03.072646 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:03.072662 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:03.073136 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:03.073320 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.073331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.075322 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.075329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.076428 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.076439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.077568 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.077577 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.077790 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.078666 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:03.078693 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:03.081973 (+ 3280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:03.085881 (+ 3908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.085887 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.085906 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.086384 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.086830 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.086837 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.086843 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.086988 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.087606 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:03.087610 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:03.087793 (+ 183us) client_negotiation.cc:769] Sending connection context
0927 22:58:03.087933 (+ 140us) client_negotiation.cc:240] Negotiation successful
0927 22:58:03.088246 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1192}
I20260927 22:58:03.088634 11301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.071290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43196 (local address 127.4.74.188:34933)
0927 22:58:03.071490 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.071496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.072130 (+ 634us) server_negotiation.cc:407] Connection header received
0927 22:58:03.072361 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.072373 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.072465 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.073485 (+ 1020us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:03.073736 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.075195 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.076568 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.077362 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.077497 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.082194 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:03.082203 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:03.082210 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:03.082252 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:03.085729 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.086518 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.086527 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.086530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.086691 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.087110 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.087115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.087119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.087420 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:03.087621 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.088044 (+ 423us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.088271 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 22:58:03.091236 10574 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.74.188:34933: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "812db8d443af4789957296a6b7a6dc53"
member_type: NON_VOTER
last_known_addr {
host: "127.4.74.188"
port: 34933
}
attrs {
promote: true
}
}
dest_uuid: "f5325a6273cb4a218f13ac8cb7dd5537"
cas_config_opid_index: 6
I20260927 22:58:03.092084 10574 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } }
I20260927 22:58:03.093303 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.054089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36562 (local address 127.4.74.129:39233)
0927 22:58:03.068338 (+ 14249us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.068347 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.068378 (+ 31us) server_negotiation.cc:407] Connection header received
0927 22:58:03.068453 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.068463 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.069983 (+ 1520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.072419 (+ 2436us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:03.072670 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.074301 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.076708 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.077725 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.077862 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.084265 (+ 6403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:03.084292 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:03.084319 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:03.084378 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:03.089182 (+ 4804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.090130 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.090140 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.090143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.090323 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.090827 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.090831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.090833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.091327 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:03.091867 (+ 540us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.092556 (+ 689us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.092871 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14016,"thread_start_us":14546,"threads_started":1}
I20260927 22:58:03.094867 11192 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 FOLLOWER]: Refusing update from remote peer f5325a6273cb4a218f13ac8cb7dd5537: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260927 22:58:03.094949 11273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:03.095187 10758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:03.095759 11303 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260927 22:58:03.098235 11273 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 FOLLOWER]: Refusing update from remote peer f5325a6273cb4a218f13ac8cb7dd5537: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260927 22:58:03.099220 11303 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:03.100935 11332 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 812db8d443af4789957296a6b7a6dc53 (127.4.74.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } } }
I20260927 22:58:03.101684 11192 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 812db8d443af4789957296a6b7a6dc53 (127.4.74.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } } }
I20260927 22:58:03.106148 11332 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.74.188:34933
I20260927 22:58:03.106711 11303 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } } }
I20260927 22:58:03.107045 11134 master.cc:537] Master@127.4.74.188:34933 shutting down...
I20260927 22:58:03.107156 11303 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:03.107324 11220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } } }
I20260927 22:58:03.107753 11220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 22:58:03.107530 11303 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } } }
I20260927 22:58:03.108083 11303 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:03.129540 11273 raft_consensus.cc:858] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } }
W20260927 22:58:03.129806 11273 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } }
W20260927 22:58:03.151978 11252 tablet.cc:2369] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:58:03.160698 11273 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7e424f35d1d146b58d3b60a554e7d0c4 (127.4.74.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:03.166860 11310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5325a6273cb4a218f13ac8cb7dd5537. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:03.167176 11310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 22:58:03.167510 11335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:03.167840 11335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260927 22:58:03.168342 11303 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } } }
I20260927 22:58:03.168634 11303 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:03.169106 11338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: NON_VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: true } } }
I20260927 22:58:03.169535 11338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260927 22:58:03.170002 10560 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260927 22:58:03.172875 11134 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
W20260927 22:58:03.173082 10560 consensus_peers.cc:487] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 -> Peer 812db8d443af4789957296a6b7a6dc53 (127.4.74.188:34933): Couldn't send request to peer 812db8d443af4789957296a6b7a6dc53. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 22:58:03.173597 11134 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260927 22:58:03.173812 11134 tablet_replica.cc:330] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: stopping tablet replica
I20260927 22:58:03.183791 11134 master.cc:552] Master@127.4.74.188:34933 shutdown complete.
I20260927 22:58:03.183921 11134 master_runner.cc:305] Clearing existing system tablet
I20260927 22:58:03.184391 11134 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:03.184514 11134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:03.189554 11134 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260927 22:58:03.192131 11341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:03.192688 11134 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:03.193114 11134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260927 22:58:03.193423 11134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal
uuid: "812db8d443af4789957296a6b7a6dc53"
format_stamp: "Formatted at 2026-09-27 22:58:02 on dist-test-slave-5pd3"
server_key: "96b638d1dfc0aefa3dabf47773f33fe4"
server_key_iv: "e4bedeb185f59ddf401a62ff4deb8c0c"
server_key_version: "encryptionkey@0"
I20260927 22:58:03.199411 11134 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:03.208516 11134 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 22:58:03.208837 11134 log.cc:1196] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260927 22:58:03.209474 11134 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Deleting consensus metadata
I20260927 22:58:03.209820 11134 master_runner.cc:315] Copying system tablet from 127.4.74.190:34435
I20260927 22:58:03.213498 11134 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: tablet copy: Beginning tablet copy session from remote peer at address 127.4.74.190:34435
I20260927 22:58:03.219738 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 10635
I20260927 22:58:03.230888 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.215084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45994 (local address 127.4.74.190:34435)
0927 22:58:03.215306 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.215312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.215616 (+ 304us) server_negotiation.cc:407] Connection header received
0927 22:58:03.215990 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.215998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.216104 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.216185 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:03.217653 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.218781 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.220998 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.221771 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.221904 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.224574 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:03.224588 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:03.224592 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:03.224631 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:03.227827 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.228530 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.228538 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.228547 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.228685 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.229093 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.229099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.229103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.229567 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:03.229773 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.230216 (+ 443us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.230481 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260927 22:58:03.231009 11347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.214605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:45994)
0927 22:58:03.215393 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:03.215422 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:03.215780 (+ 358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:03.216426 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:03.216431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:03.217057 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:03.217413 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.217465 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.219023 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.219028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.220815 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.220834 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.222229 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.222237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.222469 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.223336 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:03.223356 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:03.224342 (+ 986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:03.227978 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.227985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.227989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.228397 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.228796 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.228803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.228807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.228976 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.229748 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:03.229753 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:03.229954 (+ 201us) client_negotiation.cc:769] Sending connection context
0927 22:58:03.230107 (+ 153us) client_negotiation.cc:240] Negotiation successful
0927 22:58:03.230528 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":701,"threads_started":1}
I20260927 22:58:03.231654 10594 tablet_copy_service.cc:140] P f5325a6273cb4a218f13ac8cb7dd5537: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 812db8d443af4789957296a6b7a6dc53 ({username='oryx', principal='oryx/127.4.74.188@KRBTEST.COM'} at 127.0.0.1:45994)
I20260927 22:58:03.231850 10594 tablet_copy_service.cc:161] P f5325a6273cb4a218f13ac8cb7dd5537: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 812db8d443af4789957296a6b7a6dc53 at {username='oryx', principal='oryx/127.4.74.188@KRBTEST.COM'} at 127.0.0.1:45994: session id = 812db8d443af4789957296a6b7a6dc53-00000000000000000000000000000000
I20260927 22:58:03.236640 10594 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 22:58:03.239198 11134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 22:58:03.243924 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:39233
--local_ip_for_outbound_sockets=127.4.74.129
--tserver_master_addrs=127.4.74.190:34435
--webserver_port=33111
--webserver_interface=127.4.74.129
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab
--principal=oryx/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.74.190:34435,127.4.74.189:40471
--tserver_master_addrs=127.4.74.190:34435,127.4.74.189:40471,127.4.74.188:34933 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:58:03.247853 11134 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: tablet copy: Starting download of 0 data blocks...
I20260927 22:58:03.248229 11134 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: tablet copy: Starting download of 1 WAL segments...
I20260927 22:58:03.251763 11134 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 22:58:03.256294 10594 tablet_copy_service.cc:342] P f5325a6273cb4a218f13ac8cb7dd5537: Request end of tablet copy session 812db8d443af4789957296a6b7a6dc53-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.74.188@KRBTEST.COM'} at 127.0.0.1:45994
I20260927 22:58:03.256500 10594 tablet_copy_service.cc:433] P f5325a6273cb4a218f13ac8cb7dd5537: ending tablet copy session 812db8d443af4789957296a6b7a6dc53-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 812db8d443af4789957296a6b7a6dc53
I20260927 22:58:03.269351 11134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:03.272535 11349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:03.282102 11350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:03.282876 11352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:03.283203 11134 server_base.cc:1034] running on GCE node
I20260927 22:58:03.283759 11134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:03.285073 11134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:03.286330 11134 hybrid_clock.cc:648] HybridClock initialized: now 1790549883286277 us; error 54 us; skew 500 ppm
I20260927 22:58:03.286633 11134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:03 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549883, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:03.288924 11134 init.cc:377] Logged in from keytab as oryx/127.4.74.188@KRBTEST.COM (short username oryx)
I20260927 22:58:03.299927 11134 webserver.cc:460] Webserver started at http://127.4.74.188:38679/ using document root <none> and password file <none>
I20260927 22:58:03.300316 11134 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:03.300436 11134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:03.303239 11134 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260927 22:58:03.305375 11359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:03.305856 11134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260927 22:58:03.305999 11134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal
uuid: "812db8d443af4789957296a6b7a6dc53"
format_stamp: "Formatted at 2026-09-27 22:58:02 on dist-test-slave-5pd3"
server_key: "96b638d1dfc0aefa3dabf47773f33fe4"
server_key_iv: "e4bedeb185f59ddf401a62ff4deb8c0c"
server_key_version: "encryptionkey@0"
I20260927 22:58:03.306214 11134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:03.341946 11134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:03.345570 11134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:03.394857 11134 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.188:34933
I20260927 22:58:03.394891 11410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.188:34933 every 8 connection(s)
I20260927 22:58:03.395999 11134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
I20260927 22:58:03.406843 11411 sys_catalog.cc:263] Verifying existing consensus state
I20260927 22:58:03.412791 11411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Bootstrap starting.
I20260927 22:58:03.425597 11411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 22:58:03.426101 11411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Bootstrap complete.
I20260927 22:58:03.427037 11411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } }
I20260927 22:58:03.427331 11411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 812db8d443af4789957296a6b7a6dc53, State: Initialized, Role: LEARNER
I20260927 22:58:03.427645 11411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } }
I20260927 22:58:03.428505 11414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } } }
I20260927 22:58:03.428933 11414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: This master's current role is: LEARNER
I20260927 22:58:03.428930 11411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:58:03.440640 11425 catalog_manager.cc:1259] Loaded cluster ID: 24c7cb2cc7914dd0be03d66f9624828c
I20260927 22:58:03.440785 11425 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: loading cluster ID for follower catalog manager: success
I20260927 22:58:03.445693 11425 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: acquiring CA information for follower catalog manager: success
I20260927 22:58:03.448907 11425 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 22:58:03.718784 11348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:03.719308 11348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:03.719516 11348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:03.728423 11348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:03.728585 11348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:03.728785 11348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:58:03.739595 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } } }
I20260927 22:58:03.740002 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:03.741786 11348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab
--principal=oryx/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:39233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=33111
--tserver_master_addrs=127.4.74.190:34435,127.4.74.189:40471,127.4.74.188:34933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:03.744537 11348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:03.747164 11348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 22:58:03.756177 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.740384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.4.74.188:34933)
0927 22:58:03.749775 (+ 9391us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.749780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.749803 (+ 23us) server_negotiation.cc:407] Connection header received
0927 22:58:03.749871 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.749874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.749953 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.750089 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:03.750658 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.751781 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.754189 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.755084 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.755211 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.755429 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.755779 (+ 350us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.755893 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9221,"thread_start_us":9058,"threads_started":1}
I20260927 22:58:03.756309 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.739943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:34933 (local address 127.0.0.1:43206)
0927 22:58:03.749057 (+ 9114us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:03.749088 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:03.749345 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:03.750170 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:03.750175 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:03.750205 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:03.750452 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.750465 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.751964 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.751982 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.753950 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.753966 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.755318 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.755324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.755522 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.755618 (+ 96us) client_negotiation.cc:769] Sending connection context
0927 22:58:03.755733 (+ 115us) client_negotiation.cc:240] Negotiation successful
0927 22:58:03.755895 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8930,"thread_start_us":8791,"threads_started":1}
I20260927 22:58:03.757661 11386 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260927 22:58:03.758392 11414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5325a6273cb4a218f13ac8cb7dd5537. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: NON_VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: true } } }
I20260927 22:58:03.758836 11414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: This master's current role is: LEARNER
I20260927 22:58:03.758944 11430 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537: attempting to promote NON_VOTER 812db8d443af4789957296a6b7a6dc53 to VOTER
I20260927 22:58:03.759841 11430 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } }
I20260927 22:58:03.761871 11386 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 LEARNER]: Refusing update from remote peer f5325a6273cb4a218f13ac8cb7dd5537: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260927 22:58:03.762223 11192 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 FOLLOWER]: Refusing update from remote peer f5325a6273cb4a218f13ac8cb7dd5537: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260927 22:58:03.762506 11304 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260927 22:58:03.762940 11430 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260927 22:58:03.766037 11429 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 812db8d443af4789957296a6b7a6dc53 (127.4.74.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:03.766774 11192 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 812db8d443af4789957296a6b7a6dc53 (127.4.74.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
W20260927 22:58:03.768671 11435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:03.769083 11386 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 812db8d443af4789957296a6b7a6dc53 (127.4.74.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:03.771201 11429 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:03.771582 11429 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:03.771404 11430 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:03.771827 11430 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
W20260927 22:58:03.771863 11438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:03.771884 11429 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:03.772120 11429 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537 [sys.catalog]: This master's current role is: LEADER
W20260927 22:58:03.772737 11436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:03.773346 11348 server_base.cc:1034] running on GCE node
I20260927 22:58:03.773622 11414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:03.773537 11441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:03.773911 11414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 22:58:03.774037 11441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 22:58:03.774438 11348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:03.775745 11348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:03.777045 11348 hybrid_clock.cc:648] HybridClock initialized: now 1790549883776961 us; error 70 us; skew 500 ppm
I20260927 22:58:03.777457 11348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:03 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549883, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:03.781921 11348 init.cc:377] Logged in from keytab as oryx/127.4.74.129@KRBTEST.COM (short username oryx)
I20260927 22:58:03.783776 11348 webserver.cc:460] Webserver started at http://127.4.74.129:33111/ using document root <none> and password file <none>
I20260927 22:58:03.784581 11348 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:03.784754 11348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:03.791170 11348 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260927 22:58:03.794859 11447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:03.796805 11348 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:58:03.797065 11348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "777d65cebeda4de0baf50caf516ee3f9"
format_stamp: "Formatted at 2026-09-27 22:58:00 on dist-test-slave-5pd3"
server_key: "5b9c6eeb7bf725ec3f14a6f4fb197431"
server_key_iv: "dd4b2308d379435bb6741af84df51824"
server_key_version: "encryptionkey@0"
I20260927 22:58:03.798166 11348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:03.860141 11348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:03.866144 11348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:03.866466 11348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:03.867878 11348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:03.869827 11348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:03.869992 11348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:03.870153 11348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:03.870255 11348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:03.914567 11348 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:39233
I20260927 22:58:03.914731 11559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:39233 every 8 connection(s)
I20260927 22:58:03.916175 11348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:58:03.918678 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 11348
I20260927 22:58:03.919132 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 10769
Sep 27 22:58:03 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.130@KRBTEST.COM
Sep 27 22:58:03 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549883, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
Sep 27 22:58:03 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549883, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.188@KRBTEST.COM
Sep 27 22:58:03 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549883, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.189@KRBTEST.COM
I20260927 22:58:03.950706 11568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.921602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52788 (local address 127.4.74.130:45903)
0927 22:58:03.924352 (+ 2750us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.924369 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.925380 (+ 1011us) server_negotiation.cc:407] Connection header received
0927 22:58:03.925655 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.925666 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.926007 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.926213 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:03.929122 (+ 2909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.930850 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.933449 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.934397 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.934536 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.941300 (+ 6764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:03.941326 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:03.941361 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:03.941440 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:03.946699 (+ 5259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.947988 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.947999 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.948003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.948206 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.948729 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.948733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.948735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.949165 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:03.949538 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.949873 (+ 335us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.950227 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2436,"thread_start_us":2168,"threads_started":1}
I20260927 22:58:03.952594 10892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:03.961545 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.920966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:51999 (local address 127.4.74.188:34933)
0927 22:58:03.921300 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.921305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.923053 (+ 1748us) server_negotiation.cc:407] Connection header received
0927 22:58:03.931306 (+ 8253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.931313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.931404 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.931474 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:03.932881 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.933909 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.940570 (+ 6661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.941258 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.941340 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.952085 (+ 10745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:03.952093 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:03.952098 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:03.952151 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:03.956563 (+ 4412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.957664 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.957669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.957672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.957866 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.958614 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.958619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.958623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.958957 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:03.959168 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.960734 (+ 1566us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.961111 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260927 22:58:03.961630 11564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.920168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:37425 (local address 127.4.74.190:34435)
0927 22:58:03.921157 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.921163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.926386 (+ 5223us) server_negotiation.cc:407] Connection header received
0927 22:58:03.927839 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.927848 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.927954 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.928045 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:03.936921 (+ 8876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.938387 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.941531 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.942189 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.942349 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.948756 (+ 6407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:03.948765 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:03.948771 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:03.948830 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:03.953772 (+ 4942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.955386 (+ 1614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.955392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.955396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.955537 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.957627 (+ 2090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.957636 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.957639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.958151 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:03.958516 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.960866 (+ 2350us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.961190 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":1929,"threads_started":1}
I20260927 22:58:03.964253 11566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.921073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:34933 (local address 127.4.74.129:51999)
0927 22:58:03.922706 (+ 1633us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:03.922807 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:03.927142 (+ 4335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:03.931762 (+ 4620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:03.931769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:03.932285 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:03.932590 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.932602 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.934133 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.934138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.936662 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.936682 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.942735 (+ 6053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.942743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.943010 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.944305 (+ 1295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:03.944335 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:03.951763 (+ 7428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:03.956862 (+ 5099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.956871 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.956878 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.957472 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.958027 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.958031 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.958034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.958402 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.959151 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:03.959162 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:03.960122 (+ 960us) client_negotiation.cc:769] Sending connection context
0927 22:58:03.961822 (+ 1700us) client_negotiation.cc:240] Negotiation successful
0927 22:58:03.962204 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":35,"spinlock_wait_cycles":298624,"thread_start_us":582,"threads_started":1}
I20260927 22:58:03.964313 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.921073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.129:37425)
0927 22:58:03.926253 (+ 5180us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:03.926280 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:03.927650 (+ 1370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:03.928868 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:03.928882 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:03.929995 (+ 1113us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:03.931018 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.931049 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.938608 (+ 7559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.938614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.940648 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.940668 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.942503 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.942514 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.942963 (+ 449us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.944031 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:03.944074 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:03.948372 (+ 4298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:03.953991 (+ 5619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.954005 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.954031 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.955238 (+ 1207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.957087 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.957094 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.957101 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.957425 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.959601 (+ 2176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:03.959608 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:03.960135 (+ 527us) client_negotiation.cc:769] Sending connection context
0927 22:58:03.960740 (+ 605us) client_negotiation.cc:240] Negotiation successful
0927 22:58:03.962265 (+ 1525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4984,"mutex_wait_us":103,"spinlock_wait_cycles":96128,"thread_start_us":4579,"threads_started":1}
I20260927 22:58:03.965178 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.926791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:59317 (local address 127.4.74.189:40471)
0927 22:58:03.936156 (+ 9365us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:03.936163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:03.936188 (+ 25us) server_negotiation.cc:407] Connection header received
0927 22:58:03.936268 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:03.936275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:03.936375 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:03.936535 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:03.938784 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.939688 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.941746 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.942450 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.942583 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.949694 (+ 7111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:03.949704 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:03.949721 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:03.949775 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:03.957914 (+ 8139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.960346 (+ 2432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.960354 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.960358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.960580 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:03.962334 (+ 1754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:03.962340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:03.962343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:03.962755 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:03.962944 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:03.964446 (+ 1502us) server_negotiation.cc:299] Negotiation successful
0927 22:58:03.964738 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9196,"thread_start_us":9010,"threads_started":1}
I20260927 22:58:03.965453 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:03.926742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:40471 (local address 127.4.74.129:59317)
0927 22:58:03.935209 (+ 8467us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:03.935245 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:03.935495 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:03.936571 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:03.936576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:03.937280 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:03.937652 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.937676 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.939835 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.939842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.941507 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:03.941523 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:03.942869 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:03.942877 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:03.943067 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:03.944422 (+ 1355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:03.944441 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:03.949360 (+ 4919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:03.958116 (+ 8756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.958122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.958126 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.960116 (+ 1990us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.961962 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:03.961969 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:03.961974 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:03.962189 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:03.963014 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:03.963020 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:03.963186 (+ 166us) client_negotiation.cc:769] Sending connection context
0927 22:58:03.963343 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 22:58:03.964138 (+ 795us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8209,"mutex_wait_us":33,"spinlock_wait_cycles":205696,"thread_start_us":11684,"threads_started":1}
I20260927 22:58:03.966410 11560 heartbeater.cc:347] Connected to a master server at 127.4.74.188:34933
I20260927 22:58:03.966675 11561 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34435
I20260927 22:58:03.966820 11560 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:03.967094 11561 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:03.967679 11562 heartbeater.cc:347] Connected to a master server at 127.4.74.189:40471
I20260927 22:58:03.967849 11562 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:03.968459 11561 heartbeater.cc:511] Master 127.4.74.190:34435 requested a full tablet report, sending...
I20260927 22:58:03.968551 11562 heartbeater.cc:511] Master 127.4.74.189:40471 requested a full tablet report, sending...
I20260927 22:58:03.968580 11560 heartbeater.cc:511] Master 127.4.74.188:34933 requested a full tablet report, sending...
I20260927 22:58:03.971151 10574 ts_manager.cc:194] Re-registered known tserver with Master: 777d65cebeda4de0baf50caf516ee3f9 (127.4.74.129:39233)
I20260927 22:58:03.971747 11182 ts_manager.cc:194] Registered new tserver with Master: 777d65cebeda4de0baf50caf516ee3f9 (127.4.74.129:39233)
I20260927 22:58:03.971747 11376 ts_manager.cc:194] Registered new tserver with Master: 777d65cebeda4de0baf50caf516ee3f9 (127.4.74.129:39233)
I20260927 22:58:03.972450 10574 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:37425
I20260927 22:58:04.048611 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 10769
I20260927 22:58:04.066130 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:45903
--local_ip_for_outbound_sockets=127.4.74.130
--tserver_master_addrs=127.4.74.190:34435
--webserver_port=38807
--webserver_interface=127.4.74.130
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab
--principal=oryx/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.74.190:34435,127.4.74.189:40471
--tserver_master_addrs=127.4.74.190:34435,127.4.74.189:40471,127.4.74.188:34933 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:04.479758 11572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:04.480165 11572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:04.480348 11572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:04.488273 11572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:04.488448 11572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:04.488624 11572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:58:04.499018 11572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab
--principal=oryx/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:45903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=38807
--tserver_master_addrs=127.4.74.190:34435,127.4.74.189:40471,127.4.74.188:34933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:04.500660 11572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:04.502274 11572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:04.518026 11580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:04.518059 11577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:04.518149 11578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:04.525803 11572 server_base.cc:1034] running on GCE node
I20260927 22:58:04.526801 11572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:04.528190 11572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:04.529462 11572 hybrid_clock.cc:648] HybridClock initialized: now 1790549884529400 us; error 67 us; skew 500 ppm
I20260927 22:58:04.529938 11572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:04 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549884, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:04.541357 11572 init.cc:377] Logged in from keytab as oryx/127.4.74.130@KRBTEST.COM (short username oryx)
I20260927 22:58:04.543437 11572 webserver.cc:460] Webserver started at http://127.4.74.130:38807/ using document root <none> and password file <none>
I20260927 22:58:04.544418 11572 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:04.544588 11572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:04.551503 11572 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.004s
I20260927 22:58:04.555334 11587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:04.557400 11572 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:58:04.557657 11572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8"
format_stamp: "Formatted at 2026-09-27 22:58:00 on dist-test-slave-5pd3"
server_key: "9bb2f3bb6b10f8df9b5f7bba6f67f839"
server_key_iv: "8cc41a67a680a81095e9941faaee696e"
server_key_version: "encryptionkey@0"
I20260927 22:58:04.558925 11572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:04.584100 11572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:04.589689 11572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:04.590047 11572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:04.591431 11572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:04.593262 11572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:04.593405 11572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:04.593557 11572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:04.593653 11572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:04.636125 11572 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:45903
I20260927 22:58:04.636166 11699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:45903 every 8 connection(s)
I20260927 22:58:04.637987 11572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:58:04.645540 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 11572
I20260927 22:58:04.645843 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 10903
Sep 27 22:58:04 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549884, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
Sep 27 22:58:04 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549884, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.188@KRBTEST.COM
Sep 27 22:58:04 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.131@KRBTEST.COM
I20260927 22:58:04.678623 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:04.642375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:33983 (local address 127.4.74.190:34435)
0927 22:58:04.643149 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:04.643159 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:04.644198 (+ 1039us) server_negotiation.cc:407] Connection header received
0927 22:58:04.647944 (+ 3746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:04.647953 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:04.648064 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:04.648161 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:04.656604 (+ 8443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.658081 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.660764 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.661411 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.661541 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:04.668879 (+ 7338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:04.668888 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:04.668893 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:04.668942 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:04.673400 (+ 4458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:04.675069 (+ 1669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:04.675075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:04.675080 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:04.675274 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:04.675912 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:04.675917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:04.675921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:04.676361 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:04.676520 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:04.677761 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0927 22:58:04.678091 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":474,"threads_started":1}
Sep 27 22:58:04 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549884, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.189@KRBTEST.COM
I20260927 22:58:04.681675 11704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:04.641969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:51115 (local address 127.4.74.188:34933)
0927 22:58:04.642582 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:04.642590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:04.644674 (+ 2084us) server_negotiation.cc:407] Connection header received
0927 22:58:04.647365 (+ 2691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:04.647375 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:04.647501 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:04.647650 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:04.652839 (+ 5189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.654435 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.661767 (+ 7332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.662426 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.662643 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:04.673839 (+ 11196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:04.673848 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:04.673854 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:04.673917 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:04.677569 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:04.678553 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:04.678560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:04.678563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:04.678718 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:04.679787 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:04.679793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:04.679798 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:04.680181 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:04.680342 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:04.680923 (+ 581us) server_negotiation.cc:299] Negotiation successful
0927 22:58:04.681151 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":538,"threads_started":1}
I20260927 22:58:04.681746 11707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:04.642619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.130:33983)
0927 22:58:04.644011 (+ 1392us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:04.644061 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:04.646672 (+ 2611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:04.648776 (+ 2104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:04.648795 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:04.649919 (+ 1124us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:04.651049 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:04.651084 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.658363 (+ 7279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.658371 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:04.660489 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:04.660519 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.661921 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.661936 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:04.662391 (+ 455us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:04.663533 (+ 1142us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:04.663621 (+ 88us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:04.668510 (+ 4889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:04.673996 (+ 5486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:04.674021 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:04.674050 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:04.674879 (+ 829us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:04.675439 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:04.675445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:04.675449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:04.675774 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:04.676624 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:04.676633 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:04.677256 (+ 623us) client_negotiation.cc:769] Sending connection context
0927 22:58:04.678920 (+ 1664us) client_negotiation.cc:240] Negotiation successful
0927 22:58:04.679776 (+ 856us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"mutex_wait_us":294,"thread_start_us":651,"threads_started":1}
I20260927 22:58:04.682559 11706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:04.642619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:34933 (local address 127.4.74.130:51115)
0927 22:58:04.644407 (+ 1788us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:04.644432 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:04.646985 (+ 2553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:04.651341 (+ 4356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:04.651350 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:04.652198 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:04.652595 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:04.652610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.654691 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.654699 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:04.661270 (+ 6571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:04.661287 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.662956 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.662965 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:04.663168 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:04.669774 (+ 6606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:04.669798 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:04.673597 (+ 3799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:04.677777 (+ 4180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:04.677783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:04.677788 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:04.678394 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:04.678881 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:04.678885 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:04.678888 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:04.679653 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:04.680433 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:04.680440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:04.680651 (+ 211us) client_negotiation.cc:769] Sending connection context
0927 22:58:04.680812 (+ 161us) client_negotiation.cc:240] Negotiation successful
0927 22:58:04.681132 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":1035,"thread_start_us":586,"threads_started":1}
I20260927 22:58:04.683779 11701 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34435
I20260927 22:58:04.683911 11700 heartbeater.cc:347] Connected to a master server at 127.4.74.188:34933
I20260927 22:58:04.684393 11701 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:04.684378 11700 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:04.685684 11701 heartbeater.cc:511] Master 127.4.74.190:34435 requested a full tablet report, sending...
I20260927 22:58:04.685717 11700 heartbeater.cc:511] Master 127.4.74.188:34933 requested a full tablet report, sending...
I20260927 22:58:04.687784 10574 ts_manager.cc:194] Re-registered known tserver with Master: dd54ebab9a1a4ce09fcf9ed9c02012b8 (127.4.74.130:45903)
I20260927 22:58:04.687852 11376 ts_manager.cc:194] Registered new tserver with Master: dd54ebab9a1a4ce09fcf9ed9c02012b8 (127.4.74.130:45903)
I20260927 22:58:04.689085 10574 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:33983
I20260927 22:58:04.691134 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:04.647081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.4.74.131:39947)
0927 22:58:04.660067 (+ 12986us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:04.660076 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:04.660109 (+ 33us) server_negotiation.cc:407] Connection header received
0927 22:58:04.660182 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:04.660193 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:04.660521 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:04.663630 (+ 3109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:04.664004 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.665514 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.667269 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.668374 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.668521 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:04.677085 (+ 8564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:04.677111 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:04.677136 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:04.677188 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:04.687371 (+ 10183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:04.688492 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:04.688503 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:04.688507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:04.688760 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:04.689262 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:04.689266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:04.689268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:04.689711 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:04.690062 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:04.690485 (+ 423us) server_negotiation.cc:299] Negotiation successful
0927 22:58:04.690717 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12763,"thread_start_us":12512,"threads_started":1}
I20260927 22:58:04.692044 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:04.644981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:40471 (local address 127.4.74.130:44211)
0927 22:58:04.660677 (+ 15696us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:04.660699 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:04.660920 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:04.661476 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:04.661482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:04.662205 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:04.662591 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:04.662604 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.666795 (+ 4191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.666800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:04.668499 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:04.668516 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.673004 (+ 4488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.673010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:04.673198 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:04.678603 (+ 5405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:04.678631 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:04.682393 (+ 3762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:04.686626 (+ 4233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:04.686632 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:04.686636 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:04.687907 (+ 1271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:04.688450 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:04.688456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:04.688459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:04.688663 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:04.689462 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:04.689466 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:04.691033 (+ 1567us) client_negotiation.cc:769] Sending connection context
0927 22:58:04.691204 (+ 171us) client_negotiation.cc:240] Negotiation successful
0927 22:58:04.691585 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15516,"thread_start_us":16443,"threads_started":1}
I20260927 22:58:04.692054 11709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:04.644982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:44211 (local address 127.4.74.189:40471)
0927 22:58:04.660158 (+ 15176us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:04.660164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:04.660813 (+ 649us) server_negotiation.cc:407] Connection header received
0927 22:58:04.661070 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:04.661076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:04.661154 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:04.661237 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:04.662772 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.666591 (+ 3819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.671272 (+ 4681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:04.671970 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:04.672075 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:04.682637 (+ 10562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:04.682648 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:04.682655 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:04.682711 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:04.686453 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:04.688082 (+ 1629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:04.688088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:04.688092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:04.688283 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:04.688794 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:04.688799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:04.688803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:04.689212 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:04.689363 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:04.691400 (+ 2037us) server_negotiation.cc:299] Negotiation successful
0927 22:58:04.691695 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15051,"thread_start_us":14981,"threads_started":1}
I20260927 22:58:04.692767 11026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:04.693159 11702 heartbeater.cc:347] Connected to a master server at 127.4.74.189:40471
I20260927 22:58:04.693307 11702 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:04.693876 11702 heartbeater.cc:511] Master 127.4.74.189:40471 requested a full tablet report, sending...
I20260927 22:58:04.695067 11182 ts_manager.cc:194] Registered new tserver with Master: dd54ebab9a1a4ce09fcf9ed9c02012b8 (127.4.74.130:45903)
I20260927 22:58:04.783268 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 10903
I20260927 22:58:04.801692 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:39947
--local_ip_for_outbound_sockets=127.4.74.131
--tserver_master_addrs=127.4.74.190:34435
--webserver_port=37869
--webserver_interface=127.4.74.131
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab
--principal=oryx/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.74.190:34435,127.4.74.189:40471
--tserver_master_addrs=127.4.74.190:34435,127.4.74.189:40471,127.4.74.188:34933 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:58:04.975960 11561 heartbeater.cc:503] Master 127.4.74.190:34435 was elected leader, sending a full tablet report...
W20260927 22:58:05.219704 11713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:05.220104 11713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:05.220216 11713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:05.228659 11713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:05.228812 11713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:05.228947 11713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:58:05.240412 11713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab
--principal=oryx/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:39947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=37869
--tserver_master_addrs=127.4.74.190:34435,127.4.74.189:40471,127.4.74.188:34933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:05.242210 11713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:05.243978 11713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:05.260615 11718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:05.260802 11719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:05.261176 11721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:05.262246 11713 server_base.cc:1034] running on GCE node
I20260927 22:58:05.262913 11713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:05.264230 11713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:05.265465 11713 hybrid_clock.cc:648] HybridClock initialized: now 1790549885265390 us; error 92 us; skew 500 ppm
I20260927 22:58:05.265882 11713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:05 dist-test-slave-5pd3 krb5kdc[10529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549885, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:05.269843 11713 init.cc:377] Logged in from keytab as oryx/127.4.74.131@KRBTEST.COM (short username oryx)
I20260927 22:58:05.271986 11713 webserver.cc:460] Webserver started at http://127.4.74.131:37869/ using document root <none> and password file <none>
I20260927 22:58:05.272920 11713 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:05.273089 11713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:05.281023 11713 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260927 22:58:05.284948 11728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:05.286948 11713 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 22:58:05.287228 11713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "4b0cc13855954abc84ce0b436c7eed06"
format_stamp: "Formatted at 2026-09-27 22:58:01 on dist-test-slave-5pd3"
server_key: "6a6bd86ad1d3685ee1a1f8ca9677e624"
server_key_iv: "73ef82fca6a85350fd28ac51e09d5893"
server_key_version: "encryptionkey@0"
I20260927 22:58:05.288491 11713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:05.308756 11713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:05.313864 11713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:05.314179 11713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:05.315369 11713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:05.317034 11713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:05.317175 11713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:05.317354 11713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:05.317447 11713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:05.360085 11713 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:39947
I20260927 22:58:05.360227 11840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:39947 every 8 connection(s)
I20260927 22:58:05.361734 11713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:58:05.369812 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 11713
Sep 27 22:58:05 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549885, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.189@KRBTEST.COM
Sep 27 22:58:05 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549885, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.188@KRBTEST.COM
Sep 27 22:58:05 dist-test-slave-5pd3 krb5kdc[10529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549885, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:05.406112 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.365613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:52485 (local address 127.4.74.189:40471)
0927 22:58:05.367725 (+ 2112us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.367734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.368829 (+ 1095us) server_negotiation.cc:407] Connection header received
0927 22:58:05.377635 (+ 8806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.377641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.377736 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.377847 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:05.382420 (+ 4573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.383529 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.386224 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.386637 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.386814 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.393345 (+ 6531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:05.393355 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:05.393361 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:05.393419 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:05.398531 (+ 5112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.403147 (+ 4616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.403154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.403159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.403353 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.403947 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.403953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.403957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.404314 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:05.404469 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.405493 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.405736 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1895,"thread_start_us":1725,"threads_started":1}
I20260927 22:58:05.408515 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.366392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:40471 (local address 127.4.74.131:52485)
0927 22:58:05.368556 (+ 2164us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:05.368621 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:05.377484 (+ 8863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:05.379836 (+ 2352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:05.379844 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:05.381195 (+ 1351us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:05.382228 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.382248 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.384453 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.384459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.386066 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.386080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.386889 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.386897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.387123 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.387891 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:05.387910 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:05.391861 (+ 3951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:05.401698 (+ 9837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:05.401710 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:05.401735 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:05.403003 (+ 1268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:05.403495 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:05.403507 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:05.403511 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:05.403811 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:05.404567 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:05.404576 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:05.405049 (+ 473us) client_negotiation.cc:769] Sending connection context
0927 22:58:05.406477 (+ 1428us) client_negotiation.cc:240] Negotiation successful
0927 22:58:05.407046 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1714,"thread_start_us":1256,"threads_started":1}
I20260927 22:58:05.410836 11843 heartbeater.cc:347] Connected to a master server at 127.4.74.189:40471
I20260927 22:58:05.410971 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.365987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:41045 (local address 127.4.74.188:34933)
0927 22:58:05.367181 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.367189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.370456 (+ 3267us) server_negotiation.cc:407] Connection header received
0927 22:58:05.373218 (+ 2762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.373229 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.373356 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.373464 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:05.377656 (+ 4192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.379135 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.384393 (+ 5258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.385055 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.385187 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.394105 (+ 8918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:05.394115 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:05.394121 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:05.394188 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:05.401497 (+ 7309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.407779 (+ 6282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.407785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.407789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.407972 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.408499 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.408507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.408512 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.408910 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:05.409079 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.410298 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.410558 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1000,"thread_start_us":837,"threads_started":1}
I20260927 22:58:05.411213 11843 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:05.412000 11848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.366392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:34933 (local address 127.4.74.131:41045)
0927 22:58:05.370293 (+ 3901us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:05.370316 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:05.372599 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:05.374397 (+ 1798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:05.374415 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:05.375978 (+ 1563us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:05.377220 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.377257 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.380729 (+ 3472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.380738 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.384200 (+ 3462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.384218 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.385635 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.385643 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.385949 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.387773 (+ 1824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:05.387811 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:05.392978 (+ 5167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:05.401698 (+ 8720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:05.401710 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:05.401735 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:05.402593 (+ 858us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:05.408170 (+ 5577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:05.408174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:05.408178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:05.408357 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:05.409789 (+ 1432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:05.409793 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:05.409975 (+ 182us) client_negotiation.cc:769] Sending connection context
0927 22:58:05.411216 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0927 22:58:05.411532 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3700,"mutex_wait_us":236,"spinlock_wait_cycles":44288,"thread_start_us":1931,"threads_started":1}
I20260927 22:58:05.412253 11843 heartbeater.cc:511] Master 127.4.74.189:40471 requested a full tablet report, sending...
I20260927 22:58:05.413022 11841 heartbeater.cc:347] Connected to a master server at 127.4.74.188:34933
I20260927 22:58:05.413273 11841 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:05.413846 11841 heartbeater.cc:511] Master 127.4.74.188:34933 requested a full tablet report, sending...
I20260927 22:58:05.414970 11182 ts_manager.cc:194] Registered new tserver with Master: 4b0cc13855954abc84ce0b436c7eed06 (127.4.74.131:39947)
I20260927 22:58:05.415848 11376 ts_manager.cc:194] Registered new tserver with Master: 4b0cc13855954abc84ce0b436c7eed06 (127.4.74.131:39947)
I20260927 22:58:05.419291 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.369386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.131:38971)
0927 22:58:05.382838 (+ 13452us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:05.382864 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:05.383098 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:05.383521 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:05.383525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:05.384768 (+ 1243us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:05.385459 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.385480 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.386798 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.386804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.389077 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.389101 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.390286 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.390292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.390532 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.391469 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:05.391499 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:05.400376 (+ 8877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:05.413616 (+ 13240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:05.413629 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:05.413633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:05.414239 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:05.417241 (+ 3002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:05.417246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:05.417250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:05.417436 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:05.418302 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:05.418307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:05.418495 (+ 188us) client_negotiation.cc:769] Sending connection context
0927 22:58:05.418634 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 22:58:05.418862 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13286,"mutex_wait_us":1,"thread_start_us":13081,"threads_started":1}
I20260927 22:58:05.419639 11850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.369543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:38971 (local address 127.4.74.190:34435)
0927 22:58:05.382097 (+ 12554us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.382108 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.382992 (+ 884us) server_negotiation.cc:407] Connection header received
0927 22:58:05.383255 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.383263 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.383369 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.383495 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:05.385697 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.386603 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.389273 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.389960 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.390151 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.400807 (+ 10656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:05.400821 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:05.400830 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:05.400915 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:05.412432 (+ 11517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.416732 (+ 4300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.416743 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.416750 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.417072 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.417578 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.417586 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.417592 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.418151 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:05.418367 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.418762 (+ 395us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.419059 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12319,"thread_start_us":12590,"threads_started":1}
I20260927 22:58:05.420658 11842 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34435
I20260927 22:58:05.420814 11842 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:05.421331 11842 heartbeater.cc:511] Master 127.4.74.190:34435 requested a full tablet report, sending...
I20260927 22:58:05.422859 10574 ts_manager.cc:194] Re-registered known tserver with Master: 4b0cc13855954abc84ce0b436c7eed06 (127.4.74.131:39947)
I20260927 22:58:05.424672 10574 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:38971
I20260927 22:58:05.430480 11856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.394603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44850 (local address 127.4.74.188:34933)
0927 22:58:05.409487 (+ 14884us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.409496 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.409519 (+ 23us) server_negotiation.cc:407] Connection header received
0927 22:58:05.409589 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.409594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.409703 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.409847 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:05.413968 (+ 4121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.415209 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.416994 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.417586 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.417725 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.422756 (+ 5031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:05.422762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:05.422767 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:05.422805 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:05.426506 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.427528 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.427534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.427537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.427712 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.428201 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.428205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.428208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.428597 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:05.428809 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.429998 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.430154 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14670,"thread_start_us":15071,"threads_started":1}
I20260927 22:58:05.445405 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.419174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.4.74.189:40471)
0927 22:58:05.419384 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.419389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.431148 (+ 11759us) server_negotiation.cc:407] Connection header received
0927 22:58:05.431332 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.431337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.431438 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.431547 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:05.433484 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.434263 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.436163 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.436573 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.436726 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.440011 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:05.440016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:05.440019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:05.440057 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:05.442701 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.443413 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.443418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.443422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.443612 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.444006 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.444011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.444014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.444295 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:05.444543 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.444882 (+ 339us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.445096 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 22:58:05.446846 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.395893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.4.74.190:34435)
0927 22:58:05.417382 (+ 21489us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.417390 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.421274 (+ 3884us) server_negotiation.cc:407] Connection header received
0927 22:58:05.421620 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.421626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.421753 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.425310 (+ 3557us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:05.428832 (+ 3522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.430584 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.435586 (+ 5002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.436088 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.436280 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.440029 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:05.440035 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:05.440039 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:05.440085 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:05.443477 (+ 3392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.444388 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.444394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.444400 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.444568 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.444955 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.444961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.444966 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.445353 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:05.445680 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.446299 (+ 619us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.446527 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16939,"thread_start_us":18023,"threads_started":1}
I20260927 22:58:05.451798 10574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:58:05.459718 10574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:58:05.491062 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.482706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49670 (local address 127.4.74.131:39947)
0927 22:58:05.483427 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.483445 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.483474 (+ 29us) server_negotiation.cc:407] Connection header received
0927 22:58:05.483633 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.483642 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.483976 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.484200 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:05.484855 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.486362 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.488762 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.489866 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.490011 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.490238 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.490570 (+ 332us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.490740 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":880,"threads_started":1}
I20260927 22:58:05.491063 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.482325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:39947 (local address 127.0.0.1:49670)
0927 22:58:05.483064 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:05.483092 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:05.483271 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:05.484182 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:05.484187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:05.484222 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:05.484630 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.484670 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.486501 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.486509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.488599 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.488615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.490166 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.490172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.490378 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.490411 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 22:58:05.490524 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 22:58:05.490675 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":409,"threads_started":1}
I20260927 22:58:05.496685 11776 tablet_service.cc:1467] Processing CreateTablet for tablet 21a5540856854e1d8d88eb6b4b118746 (DEFAULT_TABLE table=test-table [id=f9315d24093d4b8f806338e5c72cb3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:05.498181 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.485501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55426 (local address 127.4.74.129:39233)
0927 22:58:05.486140 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.486149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.487154 (+ 1005us) server_negotiation.cc:407] Connection header received
0927 22:58:05.487259 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.487267 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.487580 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.487808 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:05.488887 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.492687 (+ 3800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.495154 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.496739 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.496872 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.497098 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.497755 (+ 657us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.497938 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":327,"threads_started":1}
I20260927 22:58:05.498893 11776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a5540856854e1d8d88eb6b4b118746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:05.498925 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.484411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55426)
0927 22:58:05.486897 (+ 2486us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:05.486916 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:05.487039 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:05.487851 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:05.487860 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:05.487908 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:05.488528 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.488547 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.492822 (+ 4275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.492830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.494948 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.494967 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.497238 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.497243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.497526 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.497560 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 22:58:05.498418 (+ 858us) client_negotiation.cc:240] Negotiation successful
0927 22:58:05.498591 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2353,"mutex_wait_us":76,"thread_start_us":2782,"threads_started":1}
I20260927 22:58:05.502027 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.483139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:45903 (local address 127.0.0.1:34752)
0927 22:58:05.485153 (+ 2014us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:05.485180 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:05.485306 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:05.494040 (+ 8734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:05.494046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:05.494093 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:05.494566 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.494583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.496515 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.496521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.498900 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.498924 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.501049 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.501057 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.501252 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.501285 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 22:58:05.501392 (+ 107us) client_negotiation.cc:240] Negotiation successful
0927 22:58:05.501547 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":1,"thread_start_us":803,"threads_started":1}
I20260927 22:58:05.502097 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.483459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34752 (local address 127.4.74.130:45903)
0927 22:58:05.493312 (+ 9853us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.493322 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.493355 (+ 33us) server_negotiation.cc:407] Connection header received
0927 22:58:05.493465 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.493473 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.493753 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.493980 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:05.494798 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.496327 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.499193 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.500693 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.500975 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.501228 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.501474 (+ 246us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.501701 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9654,"thread_start_us":9531,"threads_started":1}
I20260927 22:58:05.503026 11495 tablet_service.cc:1467] Processing CreateTablet for tablet 21a5540856854e1d8d88eb6b4b118746 (DEFAULT_TABLE table=test-table [id=f9315d24093d4b8f806338e5c72cb3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:05.505221 11495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a5540856854e1d8d88eb6b4b118746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:05.506332 11635 tablet_service.cc:1467] Processing CreateTablet for tablet 21a5540856854e1d8d88eb6b4b118746 (DEFAULT_TABLE table=test-table [id=f9315d24093d4b8f806338e5c72cb3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:05.507735 11635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a5540856854e1d8d88eb6b4b118746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:05.516928 11877 tablet_bootstrap.cc:492] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: Bootstrap starting.
I20260927 22:58:05.521150 11877 tablet_bootstrap.cc:654] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:05.522096 11878 tablet_bootstrap.cc:492] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: Bootstrap starting.
I20260927 22:58:05.523581 11879 tablet_bootstrap.cc:492] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Bootstrap starting.
I20260927 22:58:05.523509 11877 log.cc:824] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:05.527254 11878 tablet_bootstrap.cc:654] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:05.527456 11879 tablet_bootstrap.cc:654] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:05.528857 11879 log.cc:824] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:05.531917 11878 log.cc:824] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:05.535853 11877 tablet_bootstrap.cc:492] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: No bootstrap required, opened a new log
I20260927 22:58:05.536437 11877 ts_tablet_manager.cc:1397] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.013s
I20260927 22:58:05.538543 11879 tablet_bootstrap.cc:492] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: No bootstrap required, opened a new log
I20260927 22:58:05.538862 11879 ts_tablet_manager.cc:1397] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260927 22:58:05.551285 11878 tablet_bootstrap.cc:492] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: No bootstrap required, opened a new log
I20260927 22:58:05.551743 11878 ts_tablet_manager.cc:1397] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: Time spent bootstrapping tablet: real 0.030s user 0.012s sys 0.013s
I20260927 22:58:05.552217 11879 raft_consensus.cc:348] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:05.552111 11877 raft_consensus.cc:348] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:05.552666 11879 raft_consensus.cc:374] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:05.552673 11877 raft_consensus.cc:374] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:05.552757 11879 raft_consensus.cc:729] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd54ebab9a1a4ce09fcf9ed9c02012b8, State: Initialized, Role: FOLLOWER
I20260927 22:58:05.552766 11877 raft_consensus.cc:729] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b0cc13855954abc84ce0b436c7eed06, State: Initialized, Role: FOLLOWER
I20260927 22:58:05.553427 11879 consensus_queue.cc:260] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:05.553632 11877 consensus_queue.cc:260] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:05.554962 11701 heartbeater.cc:503] Master 127.4.74.190:34435 was elected leader, sending a full tablet report...
I20260927 22:58:05.557015 11879 ts_tablet_manager.cc:1428] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Time spent starting tablet: real 0.018s user 0.014s sys 0.001s
I20260927 22:58:05.562470 11877 ts_tablet_manager.cc:1428] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: Time spent starting tablet: real 0.026s user 0.019s sys 0.003s
I20260927 22:58:05.563218 11842 heartbeater.cc:503] Master 127.4.74.190:34435 was elected leader, sending a full tablet report...
I20260927 22:58:05.573653 11878 raft_consensus.cc:348] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:05.574156 11878 raft_consensus.cc:374] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:05.574227 11878 raft_consensus.cc:729] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 777d65cebeda4de0baf50caf516ee3f9, State: Initialized, Role: FOLLOWER
I20260927 22:58:05.575060 11878 consensus_queue.cc:260] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:05.578950 11878 ts_tablet_manager.cc:1428] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: Time spent starting tablet: real 0.027s user 0.007s sys 0.007s
W20260927 22:58:05.619259 11844 tablet.cc:2369] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 22:58:05.642455 11703 tablet.cc:2369] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 22:58:05.672678 11563 tablet.cc:2369] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:58:05.756678 11883 raft_consensus.cc:482] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:58:05.757063 11883 raft_consensus.cc:504] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:05.757121 11885 raft_consensus.cc:482] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:58:05.757529 11885 raft_consensus.cc:504] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:05.758879 11883 leader_election.cc:288] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b0cc13855954abc84ce0b436c7eed06 (127.4.74.131:39947), 777d65cebeda4de0baf50caf516ee3f9 (127.4.74.129:39233)
I20260927 22:58:05.759941 11885 leader_election.cc:288] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b0cc13855954abc84ce0b436c7eed06 (127.4.74.131:39947), dd54ebab9a1a4ce09fcf9ed9c02012b8 (127.4.74.130:45903)
I20260927 22:58:05.771687 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.759471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:33387 (local address 127.4.74.131:39947)
0927 22:58:05.759703 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.759707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.760523 (+ 816us) server_negotiation.cc:407] Connection header received
0927 22:58:05.760850 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.760859 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.760942 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.761008 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:05.762242 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.763942 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.767686 (+ 3744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.769509 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.769751 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.769974 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.771124 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.771319 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260927 22:58:05.772809 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.759517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:39947 (local address 127.4.74.130:33387)
0927 22:58:05.760405 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:05.760426 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:05.760652 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:05.761464 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:05.761472 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:05.761506 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:05.762040 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.762055 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.764241 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.764246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.767384 (+ 3138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.767406 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.770370 (+ 2964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.770380 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.770789 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.770848 (+ 59us) client_negotiation.cc:769] Sending connection context
0927 22:58:05.772085 (+ 1237us) client_negotiation.cc:240] Negotiation successful
0927 22:58:05.772317 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":467,"threads_started":1}
I20260927 22:58:05.774896 11796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a5540856854e1d8d88eb6b4b118746" candidate_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b0cc13855954abc84ce0b436c7eed06" is_pre_election: true
I20260927 22:58:05.775799 11796 raft_consensus.cc:2454] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd54ebab9a1a4ce09fcf9ed9c02012b8 in term 0.
I20260927 22:58:05.777045 11591 leader_election.cc:302] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b0cc13855954abc84ce0b436c7eed06, dd54ebab9a1a4ce09fcf9ed9c02012b8; no voters:
I20260927 22:58:05.777766 11883 raft_consensus.cc:2781] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:58:05.778008 11883 raft_consensus.cc:482] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:58:05.778129 11883 raft_consensus.cc:3037] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:05.781670 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.761008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:39679 (local address 127.4.74.129:39233)
0927 22:58:05.761320 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.761328 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.773278 (+ 11950us) server_negotiation.cc:407] Connection header received
0927 22:58:05.773630 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.773642 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.773798 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.773870 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:05.774491 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.775723 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.778686 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.780204 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.780433 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.780513 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.781155 (+ 642us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.781305 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":142}
I20260927 22:58:05.781781 11883 raft_consensus.cc:504] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:05.782442 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.760555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.4.74.130:39679)
0927 22:58:05.773143 (+ 12588us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:05.773175 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:05.773433 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:05.774107 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:05.774114 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:05.774139 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:05.774331 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.774343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.775893 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.775900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.778426 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.778475 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.780703 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.780710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.780934 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.780977 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 22:58:05.781993 (+ 1016us) client_negotiation.cc:240] Negotiation successful
0927 22:58:05.782151 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12453,"thread_start_us":12702,"threads_started":1}
I20260927 22:58:05.782621 11883 leader_election.cc:288] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [CANDIDATE]: Term 1 election: Requested vote from peers 4b0cc13855954abc84ce0b436c7eed06 (127.4.74.131:39947), 777d65cebeda4de0baf50caf516ee3f9 (127.4.74.129:39233)
I20260927 22:58:05.782852 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.760385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:39947 (local address 127.4.74.129:50255)
0927 22:58:05.772461 (+ 12076us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:05.772488 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:05.772990 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:05.774269 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:05.774330 (+ 61us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:05.774368 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:05.774901 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.774915 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.776666 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.776672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.779152 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.779163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.781237 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.781244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.781453 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.781494 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 22:58:05.782252 (+ 758us) client_negotiation.cc:240] Negotiation successful
0927 22:58:05.782409 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10919,"mutex_wait_us":1,"thread_start_us":10607,"threads_started":1}
I20260927 22:58:05.782838 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.760602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:50255 (local address 127.4.74.131:39947)
0927 22:58:05.771974 (+ 11372us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.771979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.772630 (+ 651us) server_negotiation.cc:407] Connection header received
0927 22:58:05.773774 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.773781 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.773882 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.773986 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:05.775118 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.776393 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.779312 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.780679 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.780938 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.781032 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.781674 (+ 642us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.782210 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11253,"thread_start_us":12013,"threads_started":1}
I20260927 22:58:05.783900 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.771783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:42297 (local address 127.4.74.130:45903)
0927 22:58:05.772206 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.772217 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.774329 (+ 2112us) server_negotiation.cc:407] Connection header received
0927 22:58:05.774655 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.774660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.774724 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.774780 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:05.775616 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.776711 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.780435 (+ 3724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.781935 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.782092 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.782165 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.783430 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.783633 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":158}
I20260927 22:58:05.783528 11796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a5540856854e1d8d88eb6b4b118746" candidate_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b0cc13855954abc84ce0b436c7eed06"
I20260927 22:58:05.784317 11796 raft_consensus.cc:3037] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:05.784337 11514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a5540856854e1d8d88eb6b4b118746" candidate_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777d65cebeda4de0baf50caf516ee3f9"
I20260927 22:58:05.784359 11515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a5540856854e1d8d88eb6b4b118746" candidate_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777d65cebeda4de0baf50caf516ee3f9" is_pre_election: true
I20260927 22:58:05.784868 11514 raft_consensus.cc:3037] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:05.785058 11795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a5540856854e1d8d88eb6b4b118746" candidate_uuid: "777d65cebeda4de0baf50caf516ee3f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b0cc13855954abc84ce0b436c7eed06" is_pre_election: true
I20260927 22:58:05.786659 11890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.771990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:45903 (local address 127.4.74.129:42297)
0927 22:58:05.774231 (+ 2241us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:05.774252 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:05.774456 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:05.775008 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:05.775013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:05.775038 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:05.775362 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.775375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.776953 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.776959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.780210 (+ 3251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:05.780225 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.782999 (+ 2774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.783006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:05.783209 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.783254 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 22:58:05.783369 (+ 115us) client_negotiation.cc:240] Negotiation successful
0927 22:58:05.783523 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2059,"mutex_wait_us":52,"thread_start_us":1689,"threads_started":1}
I20260927 22:58:05.787683 11655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a5540856854e1d8d88eb6b4b118746" candidate_uuid: "777d65cebeda4de0baf50caf516ee3f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" is_pre_election: true
I20260927 22:58:05.788004 11514 raft_consensus.cc:2454] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd54ebab9a1a4ce09fcf9ed9c02012b8 in term 1.
I20260927 22:58:05.788229 11655 raft_consensus.cc:2379] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 777d65cebeda4de0baf50caf516ee3f9 in current term 1: Already voted for candidate dd54ebab9a1a4ce09fcf9ed9c02012b8 in this term.
I20260927 22:58:05.788522 11796 raft_consensus.cc:2454] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd54ebab9a1a4ce09fcf9ed9c02012b8 in term 1.
I20260927 22:58:05.788686 11591 leader_election.cc:302] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 777d65cebeda4de0baf50caf516ee3f9, dd54ebab9a1a4ce09fcf9ed9c02012b8; no voters:
I20260927 22:58:05.789062 11883 raft_consensus.cc:2781] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:05.789036 11451 leader_election.cc:302] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 777d65cebeda4de0baf50caf516ee3f9; no voters: 4b0cc13855954abc84ce0b436c7eed06, dd54ebab9a1a4ce09fcf9ed9c02012b8
I20260927 22:58:05.789608 11885 raft_consensus.cc:2726] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 22:58:05.789729 11883 raft_consensus.cc:686] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 1 LEADER]: Becoming Leader. State: Replica: dd54ebab9a1a4ce09fcf9ed9c02012b8, State: Running, Role: LEADER
I20260927 22:58:05.790418 11883 consensus_queue.cc:237] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:05.796947 10573 catalog_manager.cc:5495] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd54ebab9a1a4ce09fcf9ed9c02012b8 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } health_report { overall_health: UNKNOWN } } }
I20260927 22:58:05.884984 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.878562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34758 (local address 127.4.74.130:45903)
0927 22:58:05.878746 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.878751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.878986 (+ 235us) server_negotiation.cc:407] Connection header received
0927 22:58:05.879177 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.879181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.879335 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.879502 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:58:05.880085 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.881084 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.882648 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.883085 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.883189 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.883771 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:58:05.884345 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:58:05.884467 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.884650 (+ 183us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.884722 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 22:58:05.894801 11514 raft_consensus.cc:1261] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 1 FOLLOWER]: Refusing update from remote peer dd54ebab9a1a4ce09fcf9ed9c02012b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:05.894801 11796 raft_consensus.cc:1261] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06 [term 1 FOLLOWER]: Refusing update from remote peer dd54ebab9a1a4ce09fcf9ed9c02012b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:05.895851 11891 consensus_queue.cc:1035] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:05.896349 11883 consensus_queue.cc:1035] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:05.917366 11894 mvcc.cc:205] Tried to move back new op lower bound from 7334092332611403776 to 7334092332201594880. Current Snapshot: MvccSnapshot[applied={T|T < 7334092332611403776}]
I20260927 22:58:05.938030 10574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:05.938374 10574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:05.948220 10574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5325a6273cb4a218f13ac8cb7dd5537: Sending DeleteTablet for 3 replicas of tablet 21a5540856854e1d8d88eb6b4b118746
I20260927 22:58:05.949761 11776 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a5540856854e1d8d88eb6b4b118746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:05 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:49670
I20260927 22:58:05.949859 11495 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a5540856854e1d8d88eb6b4b118746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:05 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:55426
I20260927 22:58:05.950945 11905 tablet_replica.cc:330] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: stopping tablet replica
I20260927 22:58:05.951263 11635 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a5540856854e1d8d88eb6b4b118746 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:05 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:34752
I20260927 22:58:05.951524 11905 raft_consensus.cc:2229] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:05.952273 11905 raft_consensus.cc:2258] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:05.953078 11904 tablet_replica.cc:330] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: stopping tablet replica
I20260927 22:58:05.955348 11905 ts_tablet_manager.cc:1905] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:05.956851 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 10542
I20260927 22:58:05.957636 11904 raft_consensus.cc:2229] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:05.958154 10604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:05.960175 11904 raft_consensus.cc:2258] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:05.966446 11906 tablet_replica.cc:330] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: stopping tablet replica
I20260927 22:58:05.966990 11906 raft_consensus.cc:2229] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:58:05.967212 11905 ts_tablet_manager.cc:1918] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:05.967495 11905 log.cc:1196] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/21a5540856854e1d8d88eb6b4b118746
I20260927 22:58:05.967831 11906 raft_consensus.cc:2258] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:05.968642 11905 ts_tablet_manager.cc:1939] T 21a5540856854e1d8d88eb6b4b118746 P 777d65cebeda4de0baf50caf516ee3f9: Deleting consensus metadata
I20260927 22:58:05.968775 11904 ts_tablet_manager.cc:1905] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:05.975370 11906 ts_tablet_manager.cc:1905] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:05.986016 11904 ts_tablet_manager.cc:1918] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:05.986310 11904 log.cc:1196] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/21a5540856854e1d8d88eb6b4b118746
I20260927 22:58:05.986985 11904 ts_tablet_manager.cc:1939] T 21a5540856854e1d8d88eb6b4b118746 P 4b0cc13855954abc84ce0b436c7eed06: Deleting consensus metadata
I20260927 22:58:05.990615 11906 ts_tablet_manager.cc:1918] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:05.990891 11906 log.cc:1196] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/21a5540856854e1d8d88eb6b4b118746
I20260927 22:58:05.991629 11906 ts_tablet_manager.cc:1939] T 21a5540856854e1d8d88eb6b4b118746 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Deleting consensus metadata
I20260927 22:58:06.000310 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.954582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37134 (local address 127.4.74.189:40471)
0927 22:58:05.973984 (+ 19402us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.973991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.974050 (+ 59us) server_negotiation.cc:407] Connection header received
0927 22:58:05.975665 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.975675 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.975773 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.975934 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:05.979676 (+ 3742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.980983 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.983272 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:05.984237 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:05.984494 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:05.988533 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:05.988540 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:05.988545 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:05.988594 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:05.993061 (+ 4467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.994407 (+ 1346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.994412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.994415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.994571 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:05.995070 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:05.995074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:05.995076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:05.995469 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:05.999080 (+ 3611us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:05.999281 (+ 201us) server_negotiation.cc:299] Negotiation successful
0927 22:58:05.999682 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18700,"thread_start_us":18578,"threads_started":1}
I20260927 22:58:06.017488 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:05.956659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.4.74.188:34933)
0927 22:58:05.966525 (+ 9866us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:05.966533 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:05.997007 (+ 30474us) server_negotiation.cc:407] Connection header received
0927 22:58:05.997238 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:05.997243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:05.997364 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:05.997462 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:06.000315 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:06.001551 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:06.003980 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:06.004725 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:06.005020 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:06.009312 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:06.009321 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:06.009328 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:06.009404 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:06.013566 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:06.014721 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:06.014728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:06.014732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:06.014919 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:06.015398 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:06.015403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:06.015407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:06.016068 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:06.016385 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:06.016784 (+ 399us) server_negotiation.cc:299] Negotiation successful
0927 22:58:06.017079 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9670,"thread_start_us":821,"threads_started":1}
I20260927 22:58:06.129386 10561 catalog_manager.cc:4841] TS dd54ebab9a1a4ce09fcf9ed9c02012b8 (127.4.74.130:45903): tablet 21a5540856854e1d8d88eb6b4b118746 (table test-table [id=f9315d24093d4b8f806338e5c72cb3cd]) successfully deleted
I20260927 22:58:06.130023 10561 catalog_manager.cc:4841] TS 4b0cc13855954abc84ce0b436c7eed06 (127.4.74.131:39947): tablet 21a5540856854e1d8d88eb6b4b118746 (table test-table [id=f9315d24093d4b8f806338e5c72cb3cd]) successfully deleted
I20260927 22:58:06.130251 10561 catalog_manager.cc:4841] TS 777d65cebeda4de0baf50caf516ee3f9 (127.4.74.129:39233): tablet 21a5540856854e1d8d88eb6b4b118746 (table test-table [id=f9315d24093d4b8f806338e5c72cb3cd]) successfully deleted
I20260927 22:58:06.131182 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 10542
I20260927 22:58:06.187496 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:06.166321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44860 (local address 127.4.74.188:34933)
0927 22:58:06.166580 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:06.166587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:06.168314 (+ 1727us) server_negotiation.cc:407] Connection header received
0927 22:58:06.168620 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:06.168627 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:06.168711 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:06.168786 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:06.171370 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:06.172487 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:06.174755 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:06.175437 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:06.175537 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:06.180039 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:06.180048 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:06.180054 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:06.180109 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:06.184073 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:06.185076 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:06.185083 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:06.185086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:06.185270 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:06.185754 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:06.185759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:06.185762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:06.186144 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:06.186419 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:06.186896 (+ 477us) server_negotiation.cc:299] Negotiation successful
0927 22:58:06.187169 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260927 22:58:06.200729 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:06.167725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37146 (local address 127.4.74.189:40471)
0927 22:58:06.167881 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:06.167888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:06.179902 (+ 12014us) server_negotiation.cc:407] Connection header received
0927 22:58:06.181199 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:06.181210 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:06.181367 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:06.181497 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:06.184456 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:06.185563 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:06.187822 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:06.188476 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:06.188634 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:06.193606 (+ 4972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:06.193617 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:06.193624 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:06.193687 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:06.196953 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:06.198146 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:06.198151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:06.198154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:06.198338 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:06.198862 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:06.198867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:06.198869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:06.199219 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:06.199454 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:06.200058 (+ 604us) server_negotiation.cc:299] Negotiation successful
0927 22:58:06.200303 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 22:58:06.919183 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:06.918062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.131:55685)
0927 22:58:06.918691 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:06.918862 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":313,"threads_started":1}
W20260927 22:58:06.919807 11842 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:34435 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
I20260927 22:58:06.926662 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:06.925635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.130:35301)
0927 22:58:06.926233 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:06.926412 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":279,"threads_started":1}
W20260927 22:58:06.927393 11701 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:34435 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
I20260927 22:58:06.928160 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:06.926332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.129:59381)
0927 22:58:06.926923 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:06.927057 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":283,"threads_started":1}
W20260927 22:58:06.928668 11561 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:34435 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
I20260927 22:58:07.447729 11922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5325a6273cb4a218f13ac8cb7dd5537)
I20260927 22:58:07.447774 11921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5325a6273cb4a218f13ac8cb7dd5537)
I20260927 22:58:07.448055 11922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } }
I20260927 22:58:07.448138 11921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } }
I20260927 22:58:07.449647 11922 leader_election.cc:288] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5325a6273cb4a218f13ac8cb7dd5537 (127.4.74.190:34435), 7e424f35d1d146b58d3b60a554e7d0c4 (127.4.74.189:40471)
I20260927 22:58:07.449846 11921 leader_election.cc:288] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5325a6273cb4a218f13ac8cb7dd5537 (127.4.74.190:34435), 812db8d443af4789957296a6b7a6dc53 (127.4.74.188:34933)
I20260927 22:58:07.453265 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.449805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48626)
0927 22:58:07.452623 (+ 2818us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.452896 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2605,"thread_start_us":2369,"threads_started":1}
I20260927 22:58:07.463711 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.450066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48632)
0927 22:58:07.462728 (+ 12662us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.463051 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12443,"threads_started":1}
W20260927 22:58:07.463842 11361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
W20260927 22:58:07.466135 11167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
I20260927 22:58:07.466625 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.452923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:40471 (local address 127.0.0.1:37162)
0927 22:58:07.453598 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.453623 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:07.453828 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:07.455306 (+ 1478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:07.455313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:07.455345 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:07.455678 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.455692 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.457992 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.458002 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.461955 (+ 3953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.462014 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.465127 (+ 3113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.465138 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.465526 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.465591 (+ 65us) client_negotiation.cc:769] Sending connection context
0927 22:58:07.465829 (+ 238us) client_negotiation.cc:240] Negotiation successful
0927 22:58:07.466047 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":2093,"threads_started":1}
I20260927 22:58:07.467635 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.467110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48650)
0927 22:58:07.467266 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.467378 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 22:58:07.468173 11927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.453057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37162 (local address 127.4.74.189:40471)
0927 22:58:07.454499 (+ 1442us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:07.454506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:07.454524 (+ 18us) server_negotiation.cc:407] Connection header received
0927 22:58:07.454595 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:07.454603 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:07.454728 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:07.455829 (+ 1101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:07.456093 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.457757 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.462468 (+ 4711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.464695 (+ 2227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.466944 (+ 2249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.467157 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:07.467247 (+ 90us) server_negotiation.cc:299] Negotiation successful
0927 22:58:07.467481 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1319,"thread_start_us":1094,"threads_started":1}
W20260927 22:58:07.468153 11361 leader_election.cc:334] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f5325a6273cb4a218f13ac8cb7dd5537 (127.4.74.190:34435): Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
I20260927 22:58:07.469822 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "812db8d443af4789957296a6b7a6dc53" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" is_pre_election: true
I20260927 22:58:07.470634 11192 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 812db8d443af4789957296a6b7a6dc53 in term 1.
I20260927 22:58:07.470932 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.469917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48666)
0927 22:58:07.470148 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.470641 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260927 22:58:07.471302 11167 leader_election.cc:334] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f5325a6273cb4a218f13ac8cb7dd5537 (127.4.74.190:34435): Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
I20260927 22:58:07.471715 11361 leader_election.cc:302] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e424f35d1d146b58d3b60a554e7d0c4, 812db8d443af4789957296a6b7a6dc53; no voters: f5325a6273cb4a218f13ac8cb7dd5537
I20260927 22:58:07.472198 11922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 22:58:07.472348 11922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5325a6273cb4a218f13ac8cb7dd5537)
I20260927 22:58:07.472487 11922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 1 FOLLOWER]: Advancing to term 2
I20260927 22:58:07.473893 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.455335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44874 (local address 127.4.74.188:34933)
0927 22:58:07.463529 (+ 8194us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:07.463536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:07.464530 (+ 994us) server_negotiation.cc:407] Connection header received
0927 22:58:07.464627 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:07.464634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:07.464737 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:07.464943 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:07.466265 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.467514 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.471104 (+ 3590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.472509 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.472683 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.472790 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:07.473358 (+ 568us) server_negotiation.cc:299] Negotiation successful
0927 22:58:07.473502 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8050,"thread_start_us":8222,"threads_started":1}
I20260927 22:58:07.473903 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.450314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:34933 (local address 127.0.0.1:44874)
0927 22:58:07.464113 (+ 13799us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.464133 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:07.464370 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:07.465396 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:07.465403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:07.465444 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:07.466001 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.466016 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.467778 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.467790 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.470828 (+ 3038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.470851 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.472798 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.472806 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.473054 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.473103 (+ 49us) client_negotiation.cc:769] Sending connection context
0927 22:58:07.473252 (+ 149us) client_negotiation.cc:240] Negotiation successful
0927 22:58:07.473460 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13651,"thread_start_us":12927,"threads_started":1}
I20260927 22:58:07.475026 11386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "812db8d443af4789957296a6b7a6dc53" is_pre_election: true
I20260927 22:58:07.476126 11922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } }
I20260927 22:58:07.476910 11386 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e424f35d1d146b58d3b60a554e7d0c4 in current term 2: Already voted for candidate 812db8d443af4789957296a6b7a6dc53 in this term.
I20260927 22:58:07.477051 11922 leader_election.cc:288] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [CANDIDATE]: Term 2 election: Requested vote from peers f5325a6273cb4a218f13ac8cb7dd5537 (127.4.74.190:34435), 7e424f35d1d146b58d3b60a554e7d0c4 (127.4.74.189:40471)
I20260927 22:58:07.477834 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.477215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48668)
0927 22:58:07.477489 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.477632 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 22:58:07.478085 11168 leader_election.cc:302] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e424f35d1d146b58d3b60a554e7d0c4; no voters: 812db8d443af4789957296a6b7a6dc53, f5325a6273cb4a218f13ac8cb7dd5537
I20260927 22:58:07.478256 11192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "812db8d443af4789957296a6b7a6dc53" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e424f35d1d146b58d3b60a554e7d0c4"
I20260927 22:58:07.478613 11192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 1 FOLLOWER]: Advancing to term 2
I20260927 22:58:07.479211 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.478659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48682)
0927 22:58:07.478936 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.479033 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
W20260927 22:58:07.479466 11361 leader_election.cc:334] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f5325a6273cb4a218f13ac8cb7dd5537 (127.4.74.190:34435): Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
I20260927 22:58:07.481596 11192 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 812db8d443af4789957296a6b7a6dc53 in term 2.
I20260927 22:58:07.481895 11921 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 22:58:07.482257 11361 leader_election.cc:302] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e424f35d1d146b58d3b60a554e7d0c4, 812db8d443af4789957296a6b7a6dc53; no voters: f5325a6273cb4a218f13ac8cb7dd5537
I20260927 22:58:07.482635 11922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 22:58:07.483144 11922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [term 2 LEADER]: Becoming Leader. State: Replica: 812db8d443af4789957296a6b7a6dc53, State: Running, Role: LEADER
I20260927 22:58:07.483650 11922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } }
I20260927 22:58:07.485517 11931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 812db8d443af4789957296a6b7a6dc53. Latest consensus state: current_term: 2 leader_uuid: "812db8d443af4789957296a6b7a6dc53" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:07.485912 11931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:07.486644 11933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:58:07.491011 11933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:58:07.492184 11933 catalog_manager.cc:1259] Loaded cluster ID: 24c7cb2cc7914dd0be03d66f9624828c
I20260927 22:58:07.492297 11933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:58:07.493717 11933 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:58:07.494573 11933 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Loaded TSK: 0
I20260927 22:58:07.495752 11933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 22:58:07.515182 11375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:58:07.522516 11375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:58:07.527663 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.527177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48706)
0927 22:58:07.527373 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.527461 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260927 22:58:07.528039 11192 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [term 2 FOLLOWER]: Refusing update from remote peer 812db8d443af4789957296a6b7a6dc53: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260927 22:58:07.529160 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.528666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48712)
0927 22:58:07.528859 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.528944 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260927 22:58:07.529433 11931 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260927 22:58:07.529814 11361 consensus_peers.cc:487] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 -> Peer f5325a6273cb4a218f13ac8cb7dd5537 (127.4.74.190:34435): Couldn't send request to peer f5325a6273cb4a218f13ac8cb7dd5537. Status: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 22:58:07.533049 11921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 812db8d443af4789957296a6b7a6dc53. Latest consensus state: current_term: 2 leader_uuid: "812db8d443af4789957296a6b7a6dc53" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:07.533380 11921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 22:58:07.535882 11922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "812db8d443af4789957296a6b7a6dc53" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:07.536386 11922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:07.537024 11921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "812db8d443af4789957296a6b7a6dc53" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f5325a6273cb4a218f13ac8cb7dd5537" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34435 } } peers { permanent_uuid: "7e424f35d1d146b58d3b60a554e7d0c4" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 40471 } attrs { promote: false } } peers { permanent_uuid: "812db8d443af4789957296a6b7a6dc53" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 34933 } attrs { promote: false } } }
I20260927 22:58:07.537523 11921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e424f35d1d146b58d3b60a554e7d0c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 22:58:07.556779 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.547150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:45903 (local address 127.0.0.1:34774)
0927 22:58:07.547438 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.547467 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:07.547680 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:07.549255 (+ 1575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:07.549262 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:07.549299 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:07.549516 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.549528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.551240 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.551250 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.553415 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.553424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.555856 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.555867 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.556114 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.556158 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 22:58:07.556236 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 22:58:07.556407 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 22:58:07.557950 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.547257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.4.74.130:45903)
0927 22:58:07.548773 (+ 1516us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:07.548780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:07.548807 (+ 27us) server_negotiation.cc:407] Connection header received
0927 22:58:07.548908 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:07.548916 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:07.549021 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:07.549169 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:07.549755 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.551010 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.553694 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.555594 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.557360 (+ 1766us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.557466 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:07.557535 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 22:58:07.557685 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1354,"thread_start_us":1216,"threads_started":1}
I20260927 22:58:07.559137 11635 tablet_service.cc:1467] Processing CreateTablet for tablet b86d9e81c0c545139d32fd7fa2fe7ed6 (DEFAULT_TABLE table=test-table [id=f8e63c93c5214d2ead42a966adf422bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:07.559828 11635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86d9e81c0c545139d32fd7fa2fe7ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:07.560210 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.547592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:39947 (local address 127.0.0.1:49674)
0927 22:58:07.551331 (+ 3739us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.551357 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:07.551531 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:07.552923 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:07.552928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:07.552964 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:07.553397 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.553409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.554511 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.554518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.556750 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.556772 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.559222 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.559230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.559458 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.559491 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 22:58:07.559648 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 22:58:07.559811 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244}
I20260927 22:58:07.560848 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.551052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49674 (local address 127.4.74.131:39947)
0927 22:58:07.552121 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:07.552128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:07.552153 (+ 25us) server_negotiation.cc:407] Connection header received
0927 22:58:07.552288 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:07.552293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:07.552438 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:07.552604 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:07.553603 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.554309 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.556989 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.558980 (+ 1991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.559142 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.559264 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:07.559678 (+ 414us) server_negotiation.cc:299] Negotiation successful
0927 22:58:07.560593 (+ 915us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":1194,"threads_started":1}
I20260927 22:58:07.562171 11776 tablet_service.cc:1467] Processing CreateTablet for tablet b86d9e81c0c545139d32fd7fa2fe7ed6 (DEFAULT_TABLE table=test-table [id=f8e63c93c5214d2ead42a966adf422bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:07.562709 11776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86d9e81c0c545139d32fd7fa2fe7ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:07.563792 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.552548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55440)
0927 22:58:07.556161 (+ 3613us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.556191 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:07.556369 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:07.557010 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:07.557017 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:07.557046 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:07.557431 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.557443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.558787 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.558794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.561093 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.561113 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.562705 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.562714 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.562985 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.563038 (+ 53us) client_negotiation.cc:769] Sending connection context
0927 22:58:07.563154 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 22:58:07.563322 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3427,"thread_start_us":3207,"threads_started":1}
I20260927 22:58:07.564726 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.552088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55440 (local address 127.4.74.129:39233)
0927 22:58:07.553323 (+ 1235us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:07.553331 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:07.556508 (+ 3177us) server_negotiation.cc:407] Connection header received
0927 22:58:07.556597 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:07.556603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:07.556693 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:07.556788 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:07.557636 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.558608 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.561283 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.562460 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.562603 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.562715 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:07.564170 (+ 1455us) server_negotiation.cc:299] Negotiation successful
0927 22:58:07.564374 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":1471,"threads_started":1}
I20260927 22:58:07.567979 11944 tablet_bootstrap.cc:492] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Bootstrap starting.
I20260927 22:58:07.569500 11495 tablet_service.cc:1467] Processing CreateTablet for tablet b86d9e81c0c545139d32fd7fa2fe7ed6 (DEFAULT_TABLE table=test-table [id=f8e63c93c5214d2ead42a966adf422bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:07.570009 11945 tablet_bootstrap.cc:492] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06: Bootstrap starting.
I20260927 22:58:07.570142 11495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86d9e81c0c545139d32fd7fa2fe7ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:07.570310 11944 tablet_bootstrap.cc:654] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:07.572402 11945 tablet_bootstrap.cc:654] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:07.572939 11944 tablet_bootstrap.cc:492] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8: No bootstrap required, opened a new log
I20260927 22:58:07.573223 11944 ts_tablet_manager.cc:1397] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260927 22:58:07.574110 11944 raft_consensus.cc:348] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:07.574395 11944 raft_consensus.cc:374] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:07.574498 11944 raft_consensus.cc:729] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd54ebab9a1a4ce09fcf9ed9c02012b8, State: Initialized, Role: FOLLOWER
I20260927 22:58:07.574767 11944 consensus_queue.cc:260] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:07.575628 11945 tablet_bootstrap.cc:492] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06: No bootstrap required, opened a new log
I20260927 22:58:07.575667 11944 ts_tablet_manager.cc:1428] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 22:58:07.575815 11945 ts_tablet_manager.cc:1397] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260927 22:58:07.576822 11945 raft_consensus.cc:348] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:07.577220 11945 raft_consensus.cc:374] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:07.577373 11945 raft_consensus.cc:729] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b0cc13855954abc84ce0b436c7eed06, State: Initialized, Role: FOLLOWER
I20260927 22:58:07.577659 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.576315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.130:36187)
0927 22:58:07.576819 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.577110 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"threads_started":1}
I20260927 22:58:07.577777 11945 consensus_queue.cc:260] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:07.581209 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.580617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.130:40927)
0927 22:58:07.580841 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.581047 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260927 22:58:07.581923 11700 heartbeater.cc:503] Master 127.4.74.188:34933 was elected leader, sending a full tablet report...
I20260927 22:58:07.585139 11950 tablet_bootstrap.cc:492] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9: Bootstrap starting.
I20260927 22:58:07.588850 11945 ts_tablet_manager.cc:1428] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06: Time spent starting tablet: real 0.013s user 0.000s sys 0.012s
I20260927 22:58:07.589773 11950 tablet_bootstrap.cc:654] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:07.590325 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.589006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.131:34653)
0927 22:58:07.589784 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.589988 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":463,"threads_started":1}
I20260927 22:58:07.590988 11841 heartbeater.cc:503] Master 127.4.74.188:34933 was elected leader, sending a full tablet report...
I20260927 22:58:07.592371 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.591325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.131:43081)
0927 22:58:07.592047 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.592186 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":587}
I20260927 22:58:07.594280 11950 tablet_bootstrap.cc:492] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9: No bootstrap required, opened a new log
I20260927 22:58:07.594452 11950 ts_tablet_manager.cc:1397] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260927 22:58:07.595240 11950 raft_consensus.cc:348] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:07.595453 11950 raft_consensus.cc:374] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:07.595532 11950 raft_consensus.cc:729] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 777d65cebeda4de0baf50caf516ee3f9, State: Initialized, Role: FOLLOWER
I20260927 22:58:07.595882 11950 consensus_queue.cc:260] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:07.596827 11950 ts_tablet_manager.cc:1428] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260927 22:58:07.599025 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.597725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.129:51049)
0927 22:58:07.598561 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.598697 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":547,"threads_started":1}
I20260927 22:58:07.599476 11560 heartbeater.cc:503] Master 127.4.74.188:34933 was elected leader, sending a full tablet report...
I20260927 22:58:07.600436 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.599945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.129:34845)
0927 22:58:07.600138 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.600253 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 22:58:07.619920 11951 raft_consensus.cc:482] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:58:07.620157 11951 raft_consensus.cc:504] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:07.621780 11951 leader_election.cc:288] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd54ebab9a1a4ce09fcf9ed9c02012b8 (127.4.74.130:45903), 777d65cebeda4de0baf50caf516ee3f9 (127.4.74.129:39233)
I20260927 22:58:07.630089 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.622513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:56825 (local address 127.4.74.130:45903)
0927 22:58:07.622738 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:07.622745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:07.622881 (+ 136us) server_negotiation.cc:407] Connection header received
0927 22:58:07.623190 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:07.623201 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:07.623313 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:07.623437 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:07.624039 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.625222 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.627827 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.628913 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.629143 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.629257 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:07.629598 (+ 341us) server_negotiation.cc:299] Negotiation successful
0927 22:58:07.629764 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 22:58:07.630174 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.622418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:45903 (local address 127.4.74.131:56825)
0927 22:58:07.622735 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.622758 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:07.622995 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:07.623463 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:07.623469 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:07.623502 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:07.623818 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.623831 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.625422 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.625426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.627622 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.627638 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.629156 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.629164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.629408 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.629438 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 22:58:07.629573 (+ 135us) client_negotiation.cc:240] Negotiation successful
0927 22:58:07.629764 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260927 22:58:07.630650 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.622925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.4.74.131:46989)
0927 22:58:07.623690 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.623719 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:07.623845 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:07.624356 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:07.624359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:07.624403 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:07.624624 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.624631 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.625820 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.625825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.628163 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:07.628176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.629613 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.629619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:07.629789 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.629817 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 22:58:07.629926 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 22:58:07.630085 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":435,"threads_started":1}
I20260927 22:58:07.630769 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.623117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:46989 (local address 127.4.74.129:39233)
0927 22:58:07.623359 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:07.623367 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:07.623865 (+ 498us) server_negotiation.cc:407] Connection header received
0927 22:58:07.624036 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:07.624042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:07.624171 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:07.624322 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:07.624782 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.625676 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.628424 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.629364 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.629554 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.629639 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:07.629961 (+ 322us) server_negotiation.cc:299] Negotiation successful
0927 22:58:07.630088 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 22:58:07.631265 11655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86d9e81c0c545139d32fd7fa2fe7ed6" candidate_uuid: "4b0cc13855954abc84ce0b436c7eed06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" is_pre_election: true
I20260927 22:58:07.631600 11655 raft_consensus.cc:2454] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b0cc13855954abc84ce0b436c7eed06 in term 0.
I20260927 22:58:07.631536 11514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86d9e81c0c545139d32fd7fa2fe7ed6" candidate_uuid: "4b0cc13855954abc84ce0b436c7eed06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777d65cebeda4de0baf50caf516ee3f9" is_pre_election: true
I20260927 22:58:07.631935 11514 raft_consensus.cc:2454] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b0cc13855954abc84ce0b436c7eed06 in term 0.
I20260927 22:58:07.632324 11732 leader_election.cc:302] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b0cc13855954abc84ce0b436c7eed06, dd54ebab9a1a4ce09fcf9ed9c02012b8; no voters:
I20260927 22:58:07.632879 11951 raft_consensus.cc:2781] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:58:07.633067 11951 raft_consensus.cc:482] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:58:07.633196 11951 raft_consensus.cc:3037] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:07.635460 11951 raft_consensus.cc:504] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:07.636292 11951 leader_election.cc:288] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [CANDIDATE]: Term 1 election: Requested vote from peers dd54ebab9a1a4ce09fcf9ed9c02012b8 (127.4.74.130:45903), 777d65cebeda4de0baf50caf516ee3f9 (127.4.74.129:39233)
I20260927 22:58:07.636907 11655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86d9e81c0c545139d32fd7fa2fe7ed6" candidate_uuid: "4b0cc13855954abc84ce0b436c7eed06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8"
I20260927 22:58:07.636999 11514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86d9e81c0c545139d32fd7fa2fe7ed6" candidate_uuid: "4b0cc13855954abc84ce0b436c7eed06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "777d65cebeda4de0baf50caf516ee3f9"
I20260927 22:58:07.637140 11655 raft_consensus.cc:3037] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:07.637290 11514 raft_consensus.cc:3037] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:07.639742 11655 raft_consensus.cc:2454] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b0cc13855954abc84ce0b436c7eed06 in term 1.
I20260927 22:58:07.640231 11732 leader_election.cc:302] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b0cc13855954abc84ce0b436c7eed06, dd54ebab9a1a4ce09fcf9ed9c02012b8; no voters:
I20260927 22:58:07.640549 11951 raft_consensus.cc:2781] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:07.640798 11514 raft_consensus.cc:2454] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b0cc13855954abc84ce0b436c7eed06 in term 1.
I20260927 22:58:07.641144 11951 raft_consensus.cc:686] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 1 LEADER]: Becoming Leader. State: Replica: 4b0cc13855954abc84ce0b436c7eed06, State: Running, Role: LEADER
I20260927 22:58:07.642371 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.641935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.131:39499)
0927 22:58:07.642107 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.642209 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260927 22:58:07.642442 11951 consensus_queue.cc:237] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } } peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } }
I20260927 22:58:07.650686 11375 catalog_manager.cc:5495] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b0cc13855954abc84ce0b436c7eed06 (127.4.74.131). New cstate: current_term: 1 leader_uuid: "4b0cc13855954abc84ce0b436c7eed06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b0cc13855954abc84ce0b436c7eed06" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 39947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 } health_report { overall_health: UNKNOWN } } }
I20260927 22:58:07.681668 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.674517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49686 (local address 127.4.74.131:39947)
0927 22:58:07.674741 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:07.674747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:07.674933 (+ 186us) server_negotiation.cc:407] Connection header received
0927 22:58:07.675321 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:07.675329 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:07.675475 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:07.675716 (+ 241us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:58:07.676061 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.677260 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.678700 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.679355 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.679486 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.680034 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:58:07.680934 (+ 900us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:58:07.681087 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:07.681238 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 22:58:07.681343 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 22:58:07.686920 11514 raft_consensus.cc:1261] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9 [term 1 FOLLOWER]: Refusing update from remote peer 4b0cc13855954abc84ce0b436c7eed06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:07.687093 11655 raft_consensus.cc:1261] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 1 FOLLOWER]: Refusing update from remote peer 4b0cc13855954abc84ce0b436c7eed06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:07.687923 11951 consensus_queue.cc:1035] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777d65cebeda4de0baf50caf516ee3f9" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 39233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:07.688704 11957 consensus_queue.cc:1035] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd54ebab9a1a4ce09fcf9ed9c02012b8" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 45903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 22:58:07.692911 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.692242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.130:57267)
0927 22:58:07.692539 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.692664 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260927 22:58:07.693176 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.692692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.129:45443)
0927 22:58:07.692916 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.693004 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260927 22:58:07.698338 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.697717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.129:54131)
0927 22:58:07.698017 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.698124 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260927 22:58:07.705039 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.704564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.131:37415)
0927 22:58:07.704718 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.704842 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 22:58:07.707978 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.707426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.131:57021)
0927 22:58:07.707664 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.707781 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260927 22:58:07.712845 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.712275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.4.74.130:57161)
0927 22:58:07.712533 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.712641 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260927 22:58:07.719421 11376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:07.719847 11376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:07.726387 11376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 812db8d443af4789957296a6b7a6dc53: Sending DeleteTablet for 3 replicas of tablet b86d9e81c0c545139d32fd7fa2fe7ed6
I20260927 22:58:07.727666 11635 tablet_service.cc:1514] Processing DeleteTablet for tablet b86d9e81c0c545139d32fd7fa2fe7ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:07 UTC) from {username='oryx', principal='oryx/127.4.74.188@KRBTEST.COM'} at 127.0.0.1:34774
I20260927 22:58:07.727936 11776 tablet_service.cc:1514] Processing DeleteTablet for tablet b86d9e81c0c545139d32fd7fa2fe7ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:07 UTC) from {username='oryx', principal='oryx/127.4.74.188@KRBTEST.COM'} at 127.0.0.1:49674
I20260927 22:58:07.728900 11495 tablet_service.cc:1514] Processing DeleteTablet for tablet b86d9e81c0c545139d32fd7fa2fe7ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:07 UTC) from {username='oryx', principal='oryx/127.4.74.188@KRBTEST.COM'} at 127.0.0.1:55440
I20260927 22:58:07.731048 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 11348
I20260927 22:58:07.751240 11972 tablet_replica.cc:330] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9: stopping tablet replica
I20260927 22:58:07.760247 11970 tablet_replica.cc:330] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8: stopping tablet replica
I20260927 22:58:07.751653 11972 raft_consensus.cc:2229] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:07.760511 11970 raft_consensus.cc:2229] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:07.760778 11972 raft_consensus.cc:2258] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:07.760854 11970 raft_consensus.cc:2258] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:07.762143 11970 ts_tablet_manager.cc:1905] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:07.762267 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.732242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55450 (local address 127.4.74.129:39233)
0927 22:58:07.732532 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:07.732541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:07.737466 (+ 4925us) server_negotiation.cc:407] Connection header received
0927 22:58:07.737707 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:07.737715 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:07.737821 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:07.737952 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:07.741032 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.742247 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.744601 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.745281 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.745428 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.750619 (+ 5191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:07.750646 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:07.750667 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:07.750725 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:07.756554 (+ 5829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:07.758034 (+ 1480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:07.758049 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:07.758054 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:07.758328 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:07.758936 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:07.758944 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:07.758950 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:07.759712 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:07.760173 (+ 461us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:07.761299 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0927 22:58:07.761657 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260927 22:58:07.763258 11555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:07.763746 11971 tablet_replica.cc:330] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06: stopping tablet replica
I20260927 22:58:07.764173 11971 raft_consensus.cc:2229] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:58:07.765359 11971 raft_consensus.cc:2258] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:07.770547 11970 ts_tablet_manager.cc:1918] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:07.770745 11970 log.cc:1196] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/b86d9e81c0c545139d32fd7fa2fe7ed6
I20260927 22:58:07.771654 11970 ts_tablet_manager.cc:1939] T b86d9e81c0c545139d32fd7fa2fe7ed6 P dd54ebab9a1a4ce09fcf9ed9c02012b8: Deleting consensus metadata
I20260927 22:58:07.772115 11971 ts_tablet_manager.cc:1905] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:07.773461 11363 catalog_manager.cc:4841] TS dd54ebab9a1a4ce09fcf9ed9c02012b8 (127.4.74.130:45903): tablet b86d9e81c0c545139d32fd7fa2fe7ed6 (table test-table [id=f8e63c93c5214d2ead42a966adf422bf]) successfully deleted
I20260927 22:58:07.781761 11971 ts_tablet_manager.cc:1918] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:07.781989 11971 log.cc:1196] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/b86d9e81c0c545139d32fd7fa2fe7ed6
I20260927 22:58:07.782529 11971 ts_tablet_manager.cc:1939] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 4b0cc13855954abc84ce0b436c7eed06: Deleting consensus metadata
I20260927 22:58:07.784340 11363 catalog_manager.cc:4841] TS 4b0cc13855954abc84ce0b436c7eed06 (127.4.74.131:39947): tablet b86d9e81c0c545139d32fd7fa2fe7ed6 (table test-table [id=f8e63c93c5214d2ead42a966adf422bf]) successfully deleted
I20260927 22:58:07.883517 11972 ts_tablet_manager.cc:1905] T b86d9e81c0c545139d32fd7fa2fe7ed6 P 777d65cebeda4de0baf50caf516ee3f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:07.883781 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 11348
I20260927 22:58:07.911008 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 11572
I20260927 22:58:07.911127 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.910203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48718)
0927 22:58:07.910494 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.910645 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260927 22:58:07.913062 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.912531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48728)
0927 22:58:07.912725 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.912847 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260927 22:58:07.913592 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.912869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55462)
0927 22:58:07.913338 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.913431 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:39233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315}
W20260927 22:58:07.913893 11363 catalog_manager.cc:4584] TS 777d65cebeda4de0baf50caf516ee3f9 (127.4.74.129:39233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b86d9e81c0c545139d32fd7fa2fe7ed6: Network error: Client connection negotiation failed: client connection to 127.4.74.129:39233: connect: Connection refused (error 111)
I20260927 22:58:07.927640 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.912140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.4.74.130:45903)
0927 22:58:07.912340 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:07.912346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:07.912490 (+ 144us) server_negotiation.cc:407] Connection header received
0927 22:58:07.912765 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:07.912774 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:07.912876 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:07.913008 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:07.914823 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.915702 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.917035 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:07.917564 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:07.917734 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:07.920668 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:07.920686 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:07.920705 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:07.920751 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:07.924218 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:07.925185 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:07.925201 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:07.925204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:07.925346 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:07.925789 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:07.925795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:07.925798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:07.926244 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:07.926649 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:07.926898 (+ 249us) server_negotiation.cc:299] Negotiation successful
0927 22:58:07.927167 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260927 22:58:07.928504 11695 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:07.974985 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.974265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55478)
0927 22:58:07.974635 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.974768 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:39233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203}
I20260927 22:58:07.976203 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:07.975824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55480)
0927 22:58:07.975976 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:07.976056 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:39233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 22:58:08.045240 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:08.044571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55486)
0927 22:58:08.044886 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:08.045045 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:39233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260927 22:58:08.046619 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:08.046211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55496)
0927 22:58:08.046384 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:08.046486 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:39233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260927 22:58:08.089353 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 11572
I20260927 22:58:08.120362 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 11713
I20260927 22:58:08.138666 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:08.121403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49702 (local address 127.4.74.131:39947)
0927 22:58:08.121615 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:08.121622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:08.121741 (+ 119us) server_negotiation.cc:407] Connection header received
0927 22:58:08.121990 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:08.122000 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:08.122075 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:08.122213 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:08.123892 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:08.124775 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:08.126050 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:08.126493 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:08.126636 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:08.129955 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:08.129970 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:08.129989 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:08.130039 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:08.135062 (+ 5023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:08.136224 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:08.136232 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:08.136235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:08.136407 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:08.136857 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:08.136861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:08.136863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:08.137292 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:08.137695 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:08.138020 (+ 325us) server_negotiation.cc:299] Negotiation successful
0927 22:58:08.138306 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 22:58:08.139626 11836 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:08.148685 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:08.147945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55504)
0927 22:58:08.148278 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:08.148428 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:39233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260927 22:58:08.150620 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:08.150137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55508)
0927 22:58:08.150344 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:08.150459 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:39233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260927 22:58:08.284001 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 11713
I20260927 22:58:08.310206 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 11037
I20260927 22:58:08.311805 11212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:08.322494 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:08.321705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55522)
0927 22:58:08.322104 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:08.322250 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:39233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198}
I20260927 22:58:08.323964 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:08.323440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:39233 (local address 127.0.0.1:55534)
0927 22:58:08.323715 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:08.323818 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:39233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260927 22:58:08.442962 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 11037
I20260927 22:58:08.467124 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 11134
I20260927 22:58:08.468360 11406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:08.597697 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:08.597124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34435 (local address 127.0.0.1:48736)
0927 22:58:08.597362 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:08.597503 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260927 22:58:08.597920 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 11134
2026-09-27T22:58:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11032 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:58:08 dist-test-slave-5pd3 krb5kdc[11981](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:58:08 dist-test-slave-5pd3 krb5kdc[11981](info): set up 2 sockets
Sep 27 22:58:08 dist-test-slave-5pd3 krb5kdc[11981](info): commencing operation
krb5kdc: starting...
W20260927 22:58:09.752650 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.003s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:58:09 dist-test-slave-5pd3 krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:58:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:58:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.190.keytab.
Entry for principal oryx/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.190.keytab.
I20260927 22:58:10.060405 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:35629
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.190.keytab
--principal=oryx/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:35629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:10.489609 11994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:10.489989 11994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:10.490159 11994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:10.497529 11994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:58:10.497699 11994 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 22:58:10.497776 11994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:10.497931 11994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:10.498029 11994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:58:10.498118 11994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:58:10.508270 11994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:35629
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.190.keytab
--principal=oryx/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:35629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:10.509725 11994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:10.511164 11994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:10.525539 11999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:10.525609 12000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:10.526263 12002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:10.526286 11994 server_base.cc:1034] running on GCE node
I20260927 22:58:10.527386 11994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:10.529116 11994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:10.530416 11994 hybrid_clock.cc:648] HybridClock initialized: now 1790549890530317 us; error 64 us; skew 500 ppm
I20260927 22:58:10.530869 11994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:10 dist-test-slave-5pd3 krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549890, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:10.539764 11994 init.cc:377] Logged in from keytab as oryx/127.4.74.190@KRBTEST.COM (short username oryx)
I20260927 22:58:10.541682 11994 webserver.cc:460] Webserver started at http://127.4.74.190:37235/ using document root <none> and password file <none>
I20260927 22:58:10.542520 11994 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:10.542619 11994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:10.542970 11994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:10.546229 11994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "28d475a5ed984814881646c88f9ef570"
format_stamp: "Formatted at 2026-09-27 22:58:10 on dist-test-slave-5pd3"
server_key: "90e4fee8c9deca5fc1d072edff96ba67"
server_key_iv: "715e1836b8654af8faeb03b2d668728b"
server_key_version: "encryptionkey@0"
I20260927 22:58:10.547158 11994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "28d475a5ed984814881646c88f9ef570"
format_stamp: "Formatted at 2026-09-27 22:58:10 on dist-test-slave-5pd3"
server_key: "90e4fee8c9deca5fc1d072edff96ba67"
server_key_iv: "715e1836b8654af8faeb03b2d668728b"
server_key_version: "encryptionkey@0"
I20260927 22:58:10.553239 11994 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260927 22:58:10.557746 12009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:10.559795 11994 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260927 22:58:10.560042 11994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "28d475a5ed984814881646c88f9ef570"
format_stamp: "Formatted at 2026-09-27 22:58:10 on dist-test-slave-5pd3"
server_key: "90e4fee8c9deca5fc1d072edff96ba67"
server_key_iv: "715e1836b8654af8faeb03b2d668728b"
server_key_version: "encryptionkey@0"
I20260927 22:58:10.560334 11994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:10.576133 11994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:10.581234 11994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:10.581496 11994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:10.607311 11994 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:35629
I20260927 22:58:10.607358 12070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:35629 every 8 connection(s)
I20260927 22:58:10.608873 11994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:58:10.610880 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 11994
I20260927 22:58:10.611131 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:58:10.612104 4394 external_mini_cluster.cc:1428] Setting key baced4c2e3f4e075ebfa58c7d5bc904d
I20260927 22:58:10.615131 12071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 22:58:10 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:10.632656 12071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570: Bootstrap starting.
I20260927 22:58:10.637044 12071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:10.638465 12071 log.cc:824] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:10.640414 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:10.615134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33368 (local address 127.4.74.190:35629)
0927 22:58:10.616439 (+ 1305us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:10.616453 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:10.616501 (+ 48us) server_negotiation.cc:407] Connection header received
0927 22:58:10.618271 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:10.618298 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:10.618780 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:10.619180 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:10.621336 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:10.622892 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:10.624947 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:10.625772 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:10.625942 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:10.632105 (+ 6163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:10.632114 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:10.632133 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:10.632190 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:10.635602 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:10.636434 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:10.636447 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:10.636454 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:10.636623 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:10.637025 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:10.637030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:10.637038 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:10.637728 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:10.638035 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:10.638537 (+ 502us) server_negotiation.cc:299] Negotiation successful
0927 22:58:10.638986 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":734,"threads_started":1}
I20260927 22:58:10.642681 12071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570: No bootstrap required, opened a new log
I20260927 22:58:10.651782 12071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d475a5ed984814881646c88f9ef570" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35629 } }
I20260927 22:58:10.652284 12071 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:10.652400 12071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d475a5ed984814881646c88f9ef570, State: Initialized, Role: FOLLOWER
I20260927 22:58:10.653069 12071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d475a5ed984814881646c88f9ef570" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35629 } }
I20260927 22:58:10.653352 12071 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:58:10.653509 12071 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:58:10.653718 12071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:10.656294 12071 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d475a5ed984814881646c88f9ef570" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35629 } }
I20260927 22:58:10.656924 12071 leader_election.cc:302] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28d475a5ed984814881646c88f9ef570; no voters:
I20260927 22:58:10.657651 12071 leader_election.cc:288] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:58:10.657862 12076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:10.658754 12076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [term 1 LEADER]: Becoming Leader. State: Replica: 28d475a5ed984814881646c88f9ef570, State: Running, Role: LEADER
I20260927 22:58:10.659394 12076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d475a5ed984814881646c88f9ef570" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35629 } }
I20260927 22:58:10.660310 12071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:58:10.662546 12078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d475a5ed984814881646c88f9ef570. Latest consensus state: current_term: 1 leader_uuid: "28d475a5ed984814881646c88f9ef570" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d475a5ed984814881646c88f9ef570" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35629 } } }
I20260927 22:58:10.663028 12078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:10.664090 12077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28d475a5ed984814881646c88f9ef570" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d475a5ed984814881646c88f9ef570" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35629 } } }
I20260927 22:58:10.664467 12077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:10.666946 12081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:58:10.673309 12081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:58:10.684051 12081 catalog_manager.cc:1347] Generated new cluster ID: 0e2f59b0d9c54802aac114598a7cc7e2
I20260927 22:58:10.684194 12081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:58:10.696302 12081 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:58:10.698705 12081 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:58:10.724797 12081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570: Generated new TSK 0
I20260927 22:58:10.726296 12081 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab.
Entry for principal oryx/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab.
I20260927 22:58:10.787832 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35629
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab
--principal=oryx/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:11.166754 12097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:11.167100 12097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:11.167273 12097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:11.174839 12097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:11.175027 12097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:11.175181 12097 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 22:58:11.175335 12097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:58:11.185484 12097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab
--principal=oryx/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.74.190:35629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:11.186894 12097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:11.188488 12097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:11.204249 12102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:11.208422 12105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:11.208518 12097 server_base.cc:1034] running on GCE node
W20260927 22:58:11.208921 12103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:11.209946 12097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:11.211229 12097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:11.212498 12097 hybrid_clock.cc:648] HybridClock initialized: now 1790549891212414 us; error 104 us; skew 500 ppm
I20260927 22:58:11.212913 12097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:11 dist-test-slave-5pd3 krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549891, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:11.217087 12097 init.cc:377] Logged in from keytab as oryx/127.4.74.129@KRBTEST.COM (short username oryx)
I20260927 22:58:11.219125 12097 webserver.cc:460] Webserver started at http://127.4.74.129:43489/ using document root <none> and password file <none>
I20260927 22:58:11.220127 12097 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:11.220248 12097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:11.220682 12097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:11.224444 12097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "58eaf343a3494f928b654160912dc70e"
format_stamp: "Formatted at 2026-09-27 22:58:11 on dist-test-slave-5pd3"
server_key: "55a76e5496249fce55b927efe43b9d9b"
server_key_iv: "94d56837b75537120d85cdeda928677c"
server_key_version: "encryptionkey@0"
I20260927 22:58:11.225457 12097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "58eaf343a3494f928b654160912dc70e"
format_stamp: "Formatted at 2026-09-27 22:58:11 on dist-test-slave-5pd3"
server_key: "55a76e5496249fce55b927efe43b9d9b"
server_key_iv: "94d56837b75537120d85cdeda928677c"
server_key_version: "encryptionkey@0"
I20260927 22:58:11.232930 12097 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260927 22:58:11.238081 12112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:11.240406 12097 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260927 22:58:11.240674 12097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "58eaf343a3494f928b654160912dc70e"
format_stamp: "Formatted at 2026-09-27 22:58:11 on dist-test-slave-5pd3"
server_key: "55a76e5496249fce55b927efe43b9d9b"
server_key_iv: "94d56837b75537120d85cdeda928677c"
server_key_version: "encryptionkey@0"
I20260927 22:58:11.240999 12097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:11.269447 12097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:11.274667 12097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:11.275031 12097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:11.278493 12097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:11.278640 12097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:11.278846 12097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:11.278927 12097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 22:58:11 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549891, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:11.309211 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:11.279583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:49229 (local address 127.4.74.190:35629)
0927 22:58:11.280138 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:11.280145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:11.281894 (+ 1749us) server_negotiation.cc:407] Connection header received
0927 22:58:11.285669 (+ 3775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:11.285679 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:11.285804 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:11.285987 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:11.288796 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.289800 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.293749 (+ 3949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.294255 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.294439 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:11.301180 (+ 6741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:11.301192 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:11.301200 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:11.301271 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:11.305221 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:11.306471 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:11.306479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:11.306484 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:11.306645 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:11.307265 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:11.307270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:11.307274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:11.307635 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:11.307790 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:11.308587 (+ 797us) server_negotiation.cc:299] Negotiation successful
0927 22:58:11.308841 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":274,"threads_started":1}
I20260927 22:58:11.310279 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:11.280358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35629 (local address 127.4.74.129:49229)
0927 22:58:11.281616 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:11.281683 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:11.285143 (+ 3460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:11.286375 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:11.286397 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:11.287632 (+ 1235us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:11.288565 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:11.288599 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.290053 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.290065 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:11.293483 (+ 3418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:11.293521 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.294631 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.294644 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:11.295083 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:11.296344 (+ 1261us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:11.296396 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:11.300752 (+ 4356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:11.305440 (+ 4688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:11.305456 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:11.305490 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:11.306229 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:11.306762 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:11.306768 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:11.306777 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:11.307085 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:11.307771 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:11.307784 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:11.308192 (+ 408us) client_negotiation.cc:769] Sending connection context
0927 22:58:11.308475 (+ 283us) client_negotiation.cc:240] Negotiation successful
0927 22:58:11.309041 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":521,"threads_started":1}
I20260927 22:58:11.337100 12097 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:43389
I20260927 22:58:11.337190 12232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:43389 every 8 connection(s)
I20260927 22:58:11.339082 12097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:58:11.340637 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 12097
I20260927 22:58:11.340876 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:58:11.341746 4394 external_mini_cluster.cc:1428] Setting key 7f8d447ebc0eb5e47f930dc5ce11b7b1
I20260927 22:58:11.341730 12233 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35629
I20260927 22:58:11.342017 12233 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:11.342797 12233 heartbeater.cc:511] Master 127.4.74.190:35629 requested a full tablet report, sending...
I20260927 22:58:11.345238 12026 ts_manager.cc:194] Registered new tserver with Master: 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389)
I20260927 22:58:11.347904 12026 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:49229
I20260927 22:58:11.348066 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:11.326434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:40535 (local address 127.4.74.190:35629)
0927 22:58:11.326662 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:11.326671 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:11.327508 (+ 837us) server_negotiation.cc:407] Connection header received
0927 22:58:11.327827 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:11.327834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:11.327936 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:11.328086 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:11.330426 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.331520 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.334247 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.335065 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.335433 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:11.337984 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:11.337992 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:11.337997 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:11.338045 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:11.344170 (+ 6125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:11.345125 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:11.345130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:11.345134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:11.345360 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:11.346082 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:11.346087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:11.346090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:11.346516 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:11.346771 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:11.347288 (+ 517us) server_negotiation.cc:299] Negotiation successful
0927 22:58:11.347603 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260927 22:58:11.348152 12214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:11.326350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35629 (local address 127.4.74.129:40535)
0927 22:58:11.327291 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:11.327324 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:11.327622 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:11.328280 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:11.328289 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:11.329346 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:11.330236 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:11.330251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.331764 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.331778 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:11.333968 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:11.333990 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.335478 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.335487 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:11.335728 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:11.336646 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:11.336664 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:11.337692 (+ 1028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:11.344341 (+ 6649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:11.344347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:11.344351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:11.344934 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:11.345730 (+ 796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:11.345738 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:11.345745 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:11.345941 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:11.346694 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:11.346703 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:11.346958 (+ 255us) client_negotiation.cc:769] Sending connection context
0927 22:58:11.347129 (+ 171us) client_negotiation.cc:240] Negotiation successful
0927 22:58:11.347530 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":449,"threads_started":1}
WARNING: no policy specified for oryx/127.4.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab.
Entry for principal oryx/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab.
I20260927 22:58:11.400039 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35629
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab
--principal=oryx/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:11.748732 12237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:11.749080 12237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:11.749325 12237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:11.756805 12237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:11.756978 12237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:11.757143 12237 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 22:58:11.757292 12237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:58:11.767752 12237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab
--principal=oryx/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.74.190:35629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:11.769176 12237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:11.770538 12237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:11.785769 12242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:11.786194 12243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:11.785789 12245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:11.789474 12237 server_base.cc:1034] running on GCE node
I20260927 22:58:11.790336 12237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:11.791626 12237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:11.793048 12237 hybrid_clock.cc:648] HybridClock initialized: now 1790549891792873 us; error 75 us; skew 500 ppm
I20260927 22:58:11.793485 12237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:11 dist-test-slave-5pd3 krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549891, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:11.797688 12237 init.cc:377] Logged in from keytab as oryx/127.4.74.130@KRBTEST.COM (short username oryx)
I20260927 22:58:11.799739 12237 webserver.cc:460] Webserver started at http://127.4.74.130:34303/ using document root <none> and password file <none>
I20260927 22:58:11.800552 12237 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:11.800693 12237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:11.801035 12237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:11.804224 12237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "5a92d5e4ce1c49b6a08184b000c84107"
format_stamp: "Formatted at 2026-09-27 22:58:11 on dist-test-slave-5pd3"
server_key: "b721469f594442ae7f017eea203f7946"
server_key_iv: "9c7d10b06dcf92fded4535416b03143c"
server_key_version: "encryptionkey@0"
I20260927 22:58:11.805095 12237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "5a92d5e4ce1c49b6a08184b000c84107"
format_stamp: "Formatted at 2026-09-27 22:58:11 on dist-test-slave-5pd3"
server_key: "b721469f594442ae7f017eea203f7946"
server_key_iv: "9c7d10b06dcf92fded4535416b03143c"
server_key_version: "encryptionkey@0"
I20260927 22:58:11.811303 12237 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260927 22:58:11.815614 12252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:11.817333 12237 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:58:11.817564 12237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "5a92d5e4ce1c49b6a08184b000c84107"
format_stamp: "Formatted at 2026-09-27 22:58:11 on dist-test-slave-5pd3"
server_key: "b721469f594442ae7f017eea203f7946"
server_key_iv: "9c7d10b06dcf92fded4535416b03143c"
server_key_version: "encryptionkey@0"
I20260927 22:58:11.817811 12237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:11.846028 12237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:11.850847 12237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:11.851152 12237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:11.854449 12237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:11.854575 12237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:11.854753 12237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:11.854832 12237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 22:58:11 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549891, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:11.887451 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:11.855592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:56099 (local address 127.4.74.190:35629)
0927 22:58:11.856215 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:11.856224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:11.857864 (+ 1640us) server_negotiation.cc:407] Connection header received
0927 22:58:11.860970 (+ 3106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:11.860980 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:11.861127 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:11.861317 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:11.864574 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.866081 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.869871 (+ 3790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.870659 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.870816 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:11.877940 (+ 7124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:11.877950 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:11.877957 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:11.878024 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:11.882785 (+ 4761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:11.884213 (+ 1428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:11.884218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:11.884221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:11.884468 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:11.885066 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:11.885074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:11.885078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:11.885565 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:11.885761 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:11.886706 (+ 945us) server_negotiation.cc:299] Negotiation successful
0927 22:58:11.887002 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":304,"threads_started":1}
I20260927 22:58:11.889374 12262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:11.856434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35629 (local address 127.4.74.130:56099)
0927 22:58:11.857537 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:11.857604 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:11.860401 (+ 2797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:11.861726 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:11.861748 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:11.862823 (+ 1075us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:11.864248 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:11.864283 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.866551 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.866561 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:11.869679 (+ 3118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:11.869699 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.871054 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.871062 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:11.871370 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:11.872677 (+ 1307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:11.872735 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:11.877494 (+ 4759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:11.883109 (+ 5615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:11.883123 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:11.883154 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:11.884009 (+ 855us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:11.884652 (+ 643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:11.884657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:11.884665 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:11.884913 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:11.885725 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:11.885736 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:11.886263 (+ 527us) client_negotiation.cc:769] Sending connection context
0927 22:58:11.886595 (+ 332us) client_negotiation.cc:240] Negotiation successful
0927 22:58:11.887317 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":489,"threads_started":1}
I20260927 22:58:11.917038 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:11.898050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35629 (local address 127.4.74.130:37089)
0927 22:58:11.899002 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:11.899036 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:11.899270 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:11.899862 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:11.899868 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:11.900485 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:11.900889 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:11.900901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.902262 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.902272 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:11.903473 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:11.903489 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.904752 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.904758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:11.904923 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:11.906194 (+ 1271us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:11.906228 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:11.908524 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:11.913667 (+ 5143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:11.913672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:11.913676 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:11.914163 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:11.914584 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:11.914590 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:11.914594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:11.914742 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:11.915383 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:11.915389 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:11.915715 (+ 326us) client_negotiation.cc:769] Sending connection context
0927 22:58:11.915913 (+ 198us) client_negotiation.cc:240] Negotiation successful
0927 22:58:11.916375 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":417,"threads_started":1}
I20260927 22:58:11.917996 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:11.897849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:37089 (local address 127.4.74.190:35629)
0927 22:58:11.898187 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:11.898198 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:11.899214 (+ 1016us) server_negotiation.cc:407] Connection header received
0927 22:58:11.899470 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:11.899475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:11.899621 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:11.899815 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:11.901069 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.901901 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.903708 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:11.904266 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:11.904423 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:11.908875 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:11.908881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:11.908886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:11.908934 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:11.913423 (+ 4489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:11.914316 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:11.914322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:11.914328 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:11.914465 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:11.914859 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:11.914866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:11.914870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:11.915256 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:11.915466 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:11.917462 (+ 1996us) server_negotiation.cc:299] Negotiation successful
0927 22:58:11.917699 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260927 22:58:11.924021 12237 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:42859
I20260927 22:58:11.924041 12372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:42859 every 8 connection(s)
I20260927 22:58:11.925421 12237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:58:11.929337 12373 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35629
I20260927 22:58:11.929760 12373 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:11.930523 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 12237
I20260927 22:58:11.930720 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:58:11.930948 12373 heartbeater.cc:511] Master 127.4.74.190:35629 requested a full tablet report, sending...
I20260927 22:58:11.931411 4394 external_mini_cluster.cc:1428] Setting key 9d0b6cb5736e6884552b54c00a15536c
I20260927 22:58:11.933102 12025 ts_manager.cc:194] Registered new tserver with Master: 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130:42859)
I20260927 22:58:11.934166 12025 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:56099
WARNING: no policy specified for oryx/127.4.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab.
Entry for principal oryx/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab.
I20260927 22:58:12.001535 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35629
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab
--principal=oryx/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:12.349999 12377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:12.350353 12377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:12.350526 12377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260927 22:58:12.352352 12233 heartbeater.cc:503] Master 127.4.74.190:35629 was elected leader, sending a full tablet report...
W20260927 22:58:12.358682 12377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:12.358857 12377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:12.359004 12377 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 22:58:12.359143 12377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:58:12.369570 12377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab
--principal=oryx/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.74.190:35629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:12.371009 12377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:12.372488 12377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:12.387107 12382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:12.387073 12385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:12.387158 12383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:12.388091 12377 server_base.cc:1034] running on GCE node
I20260927 22:58:12.388818 12377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:12.389988 12377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:12.391314 12377 hybrid_clock.cc:648] HybridClock initialized: now 1790549892391208 us; error 79 us; skew 500 ppm
I20260927 22:58:12.391767 12377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:12 dist-test-slave-5pd3 krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549892, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:12.395762 12377 init.cc:377] Logged in from keytab as oryx/127.4.74.131@KRBTEST.COM (short username oryx)
I20260927 22:58:12.397655 12377 webserver.cc:460] Webserver started at http://127.4.74.131:42033/ using document root <none> and password file <none>
I20260927 22:58:12.398420 12377 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:12.398563 12377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:12.398910 12377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:12.402099 12377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "d25527d7eed742e2a74c367feb69fcac"
format_stamp: "Formatted at 2026-09-27 22:58:12 on dist-test-slave-5pd3"
server_key: "b8eb2a123a9df5afc2f695c340f73081"
server_key_iv: "de9a5346111f75477be61ba4e0990a14"
server_key_version: "encryptionkey@0"
I20260927 22:58:12.402974 12377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "d25527d7eed742e2a74c367feb69fcac"
format_stamp: "Formatted at 2026-09-27 22:58:12 on dist-test-slave-5pd3"
server_key: "b8eb2a123a9df5afc2f695c340f73081"
server_key_iv: "de9a5346111f75477be61ba4e0990a14"
server_key_version: "encryptionkey@0"
I20260927 22:58:12.408958 12377 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260927 22:58:12.413676 12392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:12.415613 12377 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260927 22:58:12.415853 12377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "d25527d7eed742e2a74c367feb69fcac"
format_stamp: "Formatted at 2026-09-27 22:58:12 on dist-test-slave-5pd3"
server_key: "b8eb2a123a9df5afc2f695c340f73081"
server_key_iv: "de9a5346111f75477be61ba4e0990a14"
server_key_version: "encryptionkey@0"
I20260927 22:58:12.416111 12377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:12.440701 12377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:12.445722 12377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:12.446066 12377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:12.449236 12377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:12.449352 12377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:12.449524 12377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:12.449595 12377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 22:58:12 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549892, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:12.480098 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.450739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:39619 (local address 127.4.74.190:35629)
0927 22:58:12.451367 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:12.451374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:12.452970 (+ 1596us) server_negotiation.cc:407] Connection header received
0927 22:58:12.456680 (+ 3710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:12.456690 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:12.456854 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:12.457056 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:12.459377 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.460780 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.464210 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.465216 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.465408 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.470419 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:12.470430 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:12.470439 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:12.470505 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:12.474989 (+ 4484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:12.476385 (+ 1396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:12.476389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:12.476392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:12.476532 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:12.477573 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:12.477581 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:12.477586 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:12.477974 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:12.478147 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:12.479353 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0927 22:58:12.479655 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":294,"threads_started":1}
I20260927 22:58:12.481987 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.451535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35629 (local address 127.4.74.131:39619)
0927 22:58:12.452725 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:12.452792 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:12.456200 (+ 3408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:12.457449 (+ 1249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:12.457461 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:12.458192 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:12.459132 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:12.459154 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.461099 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.461106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:12.463936 (+ 2830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:12.463961 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.465539 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.465545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:12.465810 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.466949 (+ 1139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:12.467004 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:12.470054 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:12.475195 (+ 5141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:12.475209 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:12.475240 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:12.476174 (+ 934us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:12.476767 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:12.476775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:12.476780 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:12.477343 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:12.478127 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:12.478137 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:12.478807 (+ 670us) client_negotiation.cc:769] Sending connection context
0927 22:58:12.479262 (+ 455us) client_negotiation.cc:240] Negotiation successful
0927 22:58:12.479971 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":472,"threads_started":1}
I20260927 22:58:12.508775 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.490137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:49167 (local address 127.4.74.190:35629)
0927 22:58:12.490342 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:12.490351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:12.490998 (+ 647us) server_negotiation.cc:407] Connection header received
0927 22:58:12.491395 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:12.491403 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:12.491570 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:12.491749 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:12.493779 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.494908 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.497471 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.497962 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.498114 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.500854 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:12.500863 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:12.500870 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:12.500928 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:12.504403 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:12.506073 (+ 1670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:12.506084 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:12.506089 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:12.506371 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:12.506923 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:12.506929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:12.506933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:12.507430 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:12.507632 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:12.508063 (+ 431us) server_negotiation.cc:299] Negotiation successful
0927 22:58:12.508351 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 22:58:12.508940 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.489984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35629 (local address 127.4.74.131:49167)
0927 22:58:12.490821 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:12.490853 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:12.491151 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:12.491788 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:12.491794 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:12.492914 (+ 1120us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:12.493581 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:12.493597 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.495083 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.495089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:12.497268 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:12.497288 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.498198 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.498206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:12.498396 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.499270 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:12.499289 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:12.500570 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:12.504719 (+ 4149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:12.504731 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:12.504743 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:12.505797 (+ 1054us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:12.506546 (+ 749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:12.506555 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:12.506562 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:12.506784 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:12.507575 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:12.507580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:12.507801 (+ 221us) client_negotiation.cc:769] Sending connection context
0927 22:58:12.507966 (+ 165us) client_negotiation.cc:240] Negotiation successful
0927 22:58:12.508375 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":515,"threads_started":1}
I20260927 22:58:12.518152 12377 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:46877
I20260927 22:58:12.518198 12512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:46877 every 8 connection(s)
I20260927 22:58:12.519595 12377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:58:12.521178 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 12377
I20260927 22:58:12.521382 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:58:12.521945 4394 external_mini_cluster.cc:1428] Setting key 92c1003810b7df85e8dcbfe96add1aab
I20260927 22:58:12.523744 12513 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35629
I20260927 22:58:12.524051 12513 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:12.524886 12513 heartbeater.cc:511] Master 127.4.74.190:35629 requested a full tablet report, sending...
I20260927 22:58:12.526453 12025 ts_manager.cc:194] Registered new tserver with Master: d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877)
I20260927 22:58:12.527347 12025 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:39619
I20260927 22:58:12.538897 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 22:58:12 dist-test-slave-5pd3 krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 22:58:12 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:12 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 22:58:12.600180 12401 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:12.587606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33374 (local address 127.4.74.190:35629)
0927 22:58:12.587930 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:12.587937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:12.588193 (+ 256us) server_negotiation.cc:407] Connection header received
0927 22:58:12.588466 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:12.588471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:12.588541 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:12.588658 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:12.590059 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.590894 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.592358 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.592841 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.593007 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.599746 (+ 6739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33374: BlockingRecv error: recv got EOF from 127.0.0.1:33374 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Sep 27 22:58:12 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:12.621620 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.603919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33388 (local address 127.4.74.190:35629)
0927 22:58:12.604299 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:12.604315 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:12.604692 (+ 377us) server_negotiation.cc:407] Connection header received
0927 22:58:12.604983 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:12.604988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:12.605078 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:12.605204 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:12.606738 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.607741 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.609204 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.609805 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.610023 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.615058 (+ 5035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:12.615065 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:12.615068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:12.615112 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:12.618254 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:12.619214 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:12.619219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:12.619221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:12.619397 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:12.619882 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:12.619885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:12.619888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:12.620251 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:12.620523 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:12.620945 (+ 422us) server_negotiation.cc:299] Negotiation successful
0927 22:58:12.621246 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260927 22:58:12.626832 12025 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:58:12.634022 12025 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:58:12.678297 12532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.650709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:43389 (local address 127.0.0.1:39986)
0927 22:58:12.669093 (+ 18384us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:12.669111 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:12.669246 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:12.670004 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:12.670013 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:12.670042 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:12.670744 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:12.670772 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.672545 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.672551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:12.674708 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:12.674723 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.677186 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.677193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:12.677425 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.677476 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 22:58:12.677624 (+ 148us) client_negotiation.cc:240] Negotiation successful
0927 22:58:12.677858 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18263,"thread_start_us":17663,"threads_started":1}
I20260927 22:58:12.678370 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.651844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39986 (local address 127.4.74.129:43389)
0927 22:58:12.668082 (+ 16238us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:12.668097 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:12.669297 (+ 1200us) server_negotiation.cc:407] Connection header received
0927 22:58:12.669439 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:12.669447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:12.669760 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:12.670010 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:12.670951 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.672403 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.674953 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.676632 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.676778 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.677025 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:12.677759 (+ 734us) server_negotiation.cc:299] Negotiation successful
0927 22:58:12.677988 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16030,"thread_start_us":15869,"threads_started":1}
I20260927 22:58:12.679586 12536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.651471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58216 (local address 127.4.74.130:42859)
0927 22:58:12.658108 (+ 6637us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:12.658125 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:12.668985 (+ 10860us) server_negotiation.cc:407] Connection header received
0927 22:58:12.670393 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:12.670403 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:12.670658 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:12.670863 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:12.672018 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.673604 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.676216 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.677698 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.677839 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.678042 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:12.679010 (+ 968us) server_negotiation.cc:299] Negotiation successful
0927 22:58:12.679238 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6420,"thread_start_us":6252,"threads_started":1}
I20260927 22:58:12.679837 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.649983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:46877 (local address 127.0.0.1:48102)
0927 22:58:12.665845 (+ 15862us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:12.665930 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:12.666241 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:12.670986 (+ 4745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:12.670992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:12.671022 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:12.671640 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:12.671653 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.673780 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.673785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:12.676170 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:12.676187 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.678768 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.678777 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:12.678983 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.679020 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 22:58:12.679095 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 22:58:12.679232 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15368,"mutex_wait_us":33,"thread_start_us":16522,"threads_started":1}
I20260927 22:58:12.680444 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.650326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.4.74.131:46877)
0927 22:58:12.669878 (+ 19552us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:12.669889 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:12.669932 (+ 43us) server_negotiation.cc:407] Connection header received
0927 22:58:12.670053 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:12.670062 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:12.670407 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:12.670636 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:12.671849 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.673604 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.676389 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.677790 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.677913 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.678130 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:12.679424 (+ 1294us) server_negotiation.cc:299] Negotiation successful
0927 22:58:12.679665 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19362,"thread_start_us":19169,"threads_started":1}
I20260927 22:58:12.680577 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:12.650286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:42859 (local address 127.0.0.1:58216)
0927 22:58:12.668760 (+ 18474us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:12.668824 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:12.669133 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:12.671123 (+ 1990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:12.671127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:12.671157 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:12.671777 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:12.671789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.673771 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.673778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:12.675973 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:12.675988 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:12.678245 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:12.678251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:12.678738 (+ 487us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:12.678801 (+ 63us) client_negotiation.cc:769] Sending connection context
0927 22:58:12.678906 (+ 105us) client_negotiation.cc:240] Negotiation successful
0927 22:58:12.679097 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18304,"mutex_wait_us":70,"thread_start_us":18539,"threads_started":1}
I20260927 22:58:12.683449 12163 tablet_service.cc:1467] Processing CreateTablet for tablet d7dca6eed58e434ba64bffde621db67c (DEFAULT_TABLE table=test-table [id=ed771116539b44eb94e51962992051a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:12.684152 12443 tablet_service.cc:1467] Processing CreateTablet for tablet d7dca6eed58e434ba64bffde621db67c (DEFAULT_TABLE table=test-table [id=ed771116539b44eb94e51962992051a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:12.684070 12303 tablet_service.cc:1467] Processing CreateTablet for tablet d7dca6eed58e434ba64bffde621db67c (DEFAULT_TABLE table=test-table [id=ed771116539b44eb94e51962992051a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:12.685057 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7dca6eed58e434ba64bffde621db67c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:12.685405 12443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7dca6eed58e434ba64bffde621db67c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:12.686064 12303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7dca6eed58e434ba64bffde621db67c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:12.699473 12537 tablet_bootstrap.cc:492] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: Bootstrap starting.
I20260927 22:58:12.702244 12539 tablet_bootstrap.cc:492] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: Bootstrap starting.
I20260927 22:58:12.704291 12537 tablet_bootstrap.cc:654] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:12.706486 12539 tablet_bootstrap.cc:654] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:12.707064 12537 log.cc:824] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:12.707916 12539 log.cc:824] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:12.708949 12538 tablet_bootstrap.cc:492] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: Bootstrap starting.
I20260927 22:58:12.714051 12538 tablet_bootstrap.cc:654] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:12.714190 12537 tablet_bootstrap.cc:492] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: No bootstrap required, opened a new log
I20260927 22:58:12.714522 12537 ts_tablet_manager.cc:1397] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20260927 22:58:12.715513 12538 log.cc:824] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:12.715961 12539 tablet_bootstrap.cc:492] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: No bootstrap required, opened a new log
I20260927 22:58:12.716339 12539 ts_tablet_manager.cc:1397] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260927 22:58:12.723140 12538 tablet_bootstrap.cc:492] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: No bootstrap required, opened a new log
I20260927 22:58:12.723531 12538 ts_tablet_manager.cc:1397] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20260927 22:58:12.724634 12537 raft_consensus.cc:348] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } }
I20260927 22:58:12.725068 12537 raft_consensus.cc:374] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:12.725162 12537 raft_consensus.cc:729] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a92d5e4ce1c49b6a08184b000c84107, State: Initialized, Role: FOLLOWER
I20260927 22:58:12.725746 12537 consensus_queue.cc:260] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } }
I20260927 22:58:12.726966 12373 heartbeater.cc:503] Master 127.4.74.190:35629 was elected leader, sending a full tablet report...
I20260927 22:58:12.727528 12537 ts_tablet_manager.cc:1428] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260927 22:58:12.728163 12539 raft_consensus.cc:348] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } }
I20260927 22:58:12.728677 12539 raft_consensus.cc:374] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:12.728785 12539 raft_consensus.cc:729] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25527d7eed742e2a74c367feb69fcac, State: Initialized, Role: FOLLOWER
I20260927 22:58:12.729562 12539 consensus_queue.cc:260] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } }
I20260927 22:58:12.730893 12513 heartbeater.cc:503] Master 127.4.74.190:35629 was elected leader, sending a full tablet report...
I20260927 22:58:12.733240 12539 ts_tablet_manager.cc:1428] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260927 22:58:12.734438 12538 raft_consensus.cc:348] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } }
I20260927 22:58:12.734861 12538 raft_consensus.cc:374] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:12.734964 12538 raft_consensus.cc:729] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58eaf343a3494f928b654160912dc70e, State: Initialized, Role: FOLLOWER
I20260927 22:58:12.735751 12538 consensus_queue.cc:260] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } }
I20260927 22:58:12.737599 12538 ts_tablet_manager.cc:1428] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: Time spent starting tablet: real 0.014s user 0.007s sys 0.004s
W20260927 22:58:12.771663 12514 tablet.cc:2369] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 22:58:12.842878 12234 tablet.cc:2369] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 22:58:12.928766 12374 tablet.cc:2369] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:58:13.023015 12544 raft_consensus.cc:482] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:58:13.023407 12544 raft_consensus.cc:504] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } }
I20260927 22:58:13.025555 12544 leader_election.cc:288] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130:42859), 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389)
I20260927 22:58:13.033241 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.026018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:48867 (local address 127.4.74.129:43389)
0927 22:58:13.026238 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.026244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.027116 (+ 872us) server_negotiation.cc:407] Connection header received
0927 22:58:13.027379 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.027384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.027472 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.027649 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:13.028106 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.029054 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.031081 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.032133 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.032294 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.032359 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:13.032759 (+ 400us) server_negotiation.cc:299] Negotiation successful
0927 22:58:13.032912 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 22:58:13.033270 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.025999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:43389 (local address 127.4.74.131:48867)
0927 22:58:13.026934 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:13.026952 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:13.027137 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:13.027673 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:13.027676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:13.027697 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:13.027898 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.027906 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.029236 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.029242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.030904 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.030917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.032323 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.032330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.032541 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.032590 (+ 49us) client_negotiation.cc:769] Sending connection context
0927 22:58:13.032698 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 22:58:13.032862 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":29,"thread_start_us":611,"threads_started":1}
I20260927 22:58:13.033656 12536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.025916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:36945 (local address 127.4.74.130:42859)
0927 22:58:13.026265 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.026273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.026851 (+ 578us) server_negotiation.cc:407] Connection header received
0927 22:58:13.027079 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.027086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.027155 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.027245 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:13.028509 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.029397 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.031432 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.032479 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.032672 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.032749 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:13.033124 (+ 375us) server_negotiation.cc:299] Negotiation successful
0927 22:58:13.033272 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260927 22:58:13.033859 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.025872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:42859 (local address 127.4.74.131:36945)
0927 22:58:13.026674 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:13.026701 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:13.026912 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:13.027831 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:13.027836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:13.027861 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:13.028290 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.028304 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.029564 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.029568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.031231 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.031246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.032736 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.032742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.032930 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.032968 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 22:58:13.033070 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 22:58:13.033216 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":26,"thread_start_us":428,"threads_started":1}
I20260927 22:58:13.034686 12183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7dca6eed58e434ba64bffde621db67c" candidate_uuid: "d25527d7eed742e2a74c367feb69fcac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58eaf343a3494f928b654160912dc70e" is_pre_election: true
I20260927 22:58:13.035072 12323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7dca6eed58e434ba64bffde621db67c" candidate_uuid: "d25527d7eed742e2a74c367feb69fcac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a92d5e4ce1c49b6a08184b000c84107" is_pre_election: true
I20260927 22:58:13.035290 12183 raft_consensus.cc:2454] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d25527d7eed742e2a74c367feb69fcac in term 0.
I20260927 22:58:13.035717 12323 raft_consensus.cc:2454] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d25527d7eed742e2a74c367feb69fcac in term 0.
I20260927 22:58:13.036172 12396 leader_election.cc:302] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58eaf343a3494f928b654160912dc70e, d25527d7eed742e2a74c367feb69fcac; no voters:
I20260927 22:58:13.036732 12544 raft_consensus.cc:2781] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:58:13.036908 12544 raft_consensus.cc:482] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:58:13.037057 12544 raft_consensus.cc:3037] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:13.039768 12544 raft_consensus.cc:504] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } }
I20260927 22:58:13.040587 12544 leader_election.cc:288] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [CANDIDATE]: Term 1 election: Requested vote from peers 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130:42859), 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389)
I20260927 22:58:13.041327 12323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7dca6eed58e434ba64bffde621db67c" candidate_uuid: "d25527d7eed742e2a74c367feb69fcac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a92d5e4ce1c49b6a08184b000c84107"
I20260927 22:58:13.041326 12183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7dca6eed58e434ba64bffde621db67c" candidate_uuid: "d25527d7eed742e2a74c367feb69fcac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58eaf343a3494f928b654160912dc70e"
I20260927 22:58:13.041594 12323 raft_consensus.cc:3037] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:13.041667 12183 raft_consensus.cc:3037] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:13.045336 12183 raft_consensus.cc:2454] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d25527d7eed742e2a74c367feb69fcac in term 1.
I20260927 22:58:13.045336 12323 raft_consensus.cc:2454] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d25527d7eed742e2a74c367feb69fcac in term 1.
I20260927 22:58:13.046020 12396 leader_election.cc:302] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 58eaf343a3494f928b654160912dc70e, 5a92d5e4ce1c49b6a08184b000c84107, d25527d7eed742e2a74c367feb69fcac; no voters:
I20260927 22:58:13.046547 12544 raft_consensus.cc:2781] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:13.047308 12544 raft_consensus.cc:686] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 1 LEADER]: Becoming Leader. State: Replica: d25527d7eed742e2a74c367feb69fcac, State: Running, Role: LEADER
I20260927 22:58:13.047935 12544 consensus_queue.cc:237] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } }
I20260927 22:58:13.053781 12026 catalog_manager.cc:5495] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac reported cstate change: term changed from 0 to 1, leader changed from <none> to d25527d7eed742e2a74c367feb69fcac (127.4.74.131). New cstate: current_term: 1 leader_uuid: "d25527d7eed742e2a74c367feb69fcac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } health_report { overall_health: UNKNOWN } } }
I20260927 22:58:13.129132 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.122333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48114 (local address 127.4.74.131:46877)
0927 22:58:13.122620 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.122637 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.123301 (+ 664us) server_negotiation.cc:407] Connection header received
0927 22:58:13.123644 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.123649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.123754 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.123933 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:58:13.124334 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.125139 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.126621 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.127110 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.127228 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.127792 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:58:13.128493 (+ 701us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:58:13.128623 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:13.128793 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 22:58:13.128881 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 22:58:13.139361 12323 raft_consensus.cc:1261] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Refusing update from remote peer d25527d7eed742e2a74c367feb69fcac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:13.139360 12183 raft_consensus.cc:1261] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Refusing update from remote peer d25527d7eed742e2a74c367feb69fcac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:13.140928 12548 consensus_queue.cc:1035] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:13.141403 12544 consensus_queue.cc:1035] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [LEADER]: Connected to new peer: Peer: permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:13.177177 12026 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:13.177510 12026 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:13.181964 12026 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570: Sending DeleteTablet for 3 replicas of tablet d7dca6eed58e434ba64bffde621db67c
I20260927 22:58:13.183434 12303 tablet_service.cc:1514] Processing DeleteTablet for tablet d7dca6eed58e434ba64bffde621db67c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:13 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:58216
I20260927 22:58:13.183452 12443 tablet_service.cc:1514] Processing DeleteTablet for tablet d7dca6eed58e434ba64bffde621db67c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:13 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:48102
I20260927 22:58:13.183774 12163 tablet_service.cc:1514] Processing DeleteTablet for tablet d7dca6eed58e434ba64bffde621db67c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:13 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:39986
I20260927 22:58:13.186846 12026 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:58:13.187631 12026 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:58:13.201424 12302 tablet_service.cc:1467] Processing CreateTablet for tablet ba5a5ae5dbef4e1ca409d2b5630302b9 (DEFAULT_TABLE table=test-table [id=7d5420fda4174beeb31db2d040eddafe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:13.202140 12302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba5a5ae5dbef4e1ca409d2b5630302b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:13.202589 12442 tablet_service.cc:1467] Processing CreateTablet for tablet ba5a5ae5dbef4e1ca409d2b5630302b9 (DEFAULT_TABLE table=test-table [id=7d5420fda4174beeb31db2d040eddafe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:13.205251 12442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba5a5ae5dbef4e1ca409d2b5630302b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:13.205631 12162 tablet_service.cc:1467] Processing CreateTablet for tablet ba5a5ae5dbef4e1ca409d2b5630302b9 (DEFAULT_TABLE table=test-table [id=7d5420fda4174beeb31db2d040eddafe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:13.206245 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba5a5ae5dbef4e1ca409d2b5630302b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:13.210508 12537 tablet_bootstrap.cc:492] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107: Bootstrap starting.
I20260927 22:58:13.212357 12537 tablet_bootstrap.cc:654] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:13.213297 12538 tablet_bootstrap.cc:492] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e: Bootstrap starting.
I20260927 22:58:13.215265 12538 tablet_bootstrap.cc:654] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:13.217911 12539 tablet_bootstrap.cc:492] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac: Bootstrap starting.
I20260927 22:58:13.220999 12539 tablet_bootstrap.cc:654] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:13.231431 12537 tablet_bootstrap.cc:492] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107: No bootstrap required, opened a new log
I20260927 22:58:13.231667 12537 ts_tablet_manager.cc:1397] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.007s
I20260927 22:58:13.232692 12537 raft_consensus.cc:348] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.233019 12537 raft_consensus.cc:374] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:13.233153 12537 raft_consensus.cc:729] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a92d5e4ce1c49b6a08184b000c84107, State: Initialized, Role: FOLLOWER
I20260927 22:58:13.233517 12537 consensus_queue.cc:260] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.234383 12537 ts_tablet_manager.cc:1428] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260927 22:58:13.235569 12538 tablet_bootstrap.cc:492] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e: No bootstrap required, opened a new log
I20260927 22:58:13.235746 12538 ts_tablet_manager.cc:1397] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.005s
I20260927 22:58:13.236667 12538 raft_consensus.cc:348] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.236924 12538 raft_consensus.cc:374] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:13.237049 12538 raft_consensus.cc:729] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58eaf343a3494f928b654160912dc70e, State: Initialized, Role: FOLLOWER
I20260927 22:58:13.237470 12564 tablet_replica.cc:330] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: stopping tablet replica
I20260927 22:58:13.237360 12538 consensus_queue.cc:260] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.238147 12539 tablet_bootstrap.cc:492] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac: No bootstrap required, opened a new log
I20260927 22:58:13.238181 12538 ts_tablet_manager.cc:1428] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 22:58:13.238440 12539 ts_tablet_manager.cc:1397] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.004s
I20260927 22:58:13.239454 12562 tablet_replica.cc:330] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: stopping tablet replica
I20260927 22:58:13.239662 12564 raft_consensus.cc:2229] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:13.240226 12564 raft_consensus.cc:2258] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:13.240199 12562 raft_consensus.cc:2229] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:58:13.241112 12562 raft_consensus.cc:2258] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:13.242445 12564 ts_tablet_manager.cc:1905] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:13.242442 12539 raft_consensus.cc:348] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.242918 12539 raft_consensus.cc:374] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:13.243094 12539 raft_consensus.cc:729] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25527d7eed742e2a74c367feb69fcac, State: Initialized, Role: FOLLOWER
I20260927 22:58:13.243469 12539 consensus_queue.cc:260] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.244289 12539 ts_tablet_manager.cc:1428] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260927 22:58:13.246790 12563 tablet_replica.cc:330] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: stopping tablet replica
I20260927 22:58:13.247414 12563 raft_consensus.cc:2229] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:13.248149 12563 raft_consensus.cc:2258] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:13.246320 12562 ts_tablet_manager.cc:1905] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:13.253399 12563 ts_tablet_manager.cc:1905] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:13.255483 12564 ts_tablet_manager.cc:1918] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:13.256765 12562 ts_tablet_manager.cc:1918] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:13.257066 12562 log.cc:1196] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/d7dca6eed58e434ba64bffde621db67c
I20260927 22:58:13.257216 12564 log.cc:1196] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/d7dca6eed58e434ba64bffde621db67c
I20260927 22:58:13.257956 12564 ts_tablet_manager.cc:1939] T d7dca6eed58e434ba64bffde621db67c P 58eaf343a3494f928b654160912dc70e: Deleting consensus metadata
I20260927 22:58:13.257967 12562 ts_tablet_manager.cc:1939] T d7dca6eed58e434ba64bffde621db67c P d25527d7eed742e2a74c367feb69fcac: Deleting consensus metadata
I20260927 22:58:13.260071 12013 catalog_manager.cc:4841] TS 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389): tablet d7dca6eed58e434ba64bffde621db67c (table test-table [id=ed771116539b44eb94e51962992051a0]) successfully deleted
I20260927 22:58:13.260326 12011 catalog_manager.cc:4841] TS d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877): tablet d7dca6eed58e434ba64bffde621db67c (table test-table [id=ed771116539b44eb94e51962992051a0]) successfully deleted
I20260927 22:58:13.261621 12563 ts_tablet_manager.cc:1918] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:13.261834 12563 log.cc:1196] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/d7dca6eed58e434ba64bffde621db67c
I20260927 22:58:13.262354 12563 ts_tablet_manager.cc:1939] T d7dca6eed58e434ba64bffde621db67c P 5a92d5e4ce1c49b6a08184b000c84107: Deleting consensus metadata
I20260927 22:58:13.264339 12010 catalog_manager.cc:4841] TS 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130:42859): tablet d7dca6eed58e434ba64bffde621db67c (table test-table [id=ed771116539b44eb94e51962992051a0]) successfully deleted
I20260927 22:58:13.389657 12543 raft_consensus.cc:482] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:58:13.389940 12543 raft_consensus.cc:504] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.392016 12543 leader_election.cc:288] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389), d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877)
I20260927 22:58:13.401207 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.392659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:39133 (local address 127.4.74.129:43389)
0927 22:58:13.392869 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.392876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.394369 (+ 1493us) server_negotiation.cc:407] Connection header received
0927 22:58:13.394554 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.394560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.394653 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.394756 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:13.395647 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.396754 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.398930 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.400027 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.400217 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.400308 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:13.400701 (+ 393us) server_negotiation.cc:299] Negotiation successful
0927 22:58:13.400858 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 22:58:13.401750 12569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.393344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:43389 (local address 127.4.74.130:39133)
0927 22:58:13.394226 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:13.394244 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:13.394393 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:13.394805 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:13.394816 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:13.394879 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:13.395430 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.395443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.396918 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.396923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.398748 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.398759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.400268 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.400276 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.400485 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.400527 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 22:58:13.400644 (+ 117us) client_negotiation.cc:240] Negotiation successful
0927 22:58:13.400821 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":37,"spinlock_wait_cycles":7552,"thread_start_us":482,"threads_started":1}
I20260927 22:58:13.402236 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.392639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:56141 (local address 127.4.74.131:46877)
0927 22:58:13.392933 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.392941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.393926 (+ 985us) server_negotiation.cc:407] Connection header received
0927 22:58:13.394290 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.394298 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.394424 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.394582 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:13.395342 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.396698 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.399597 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.400937 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.401088 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.401156 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:13.401765 (+ 609us) server_negotiation.cc:299] Negotiation successful
0927 22:58:13.401907 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260927 22:58:13.402339 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.392780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:46877 (local address 127.4.74.130:56141)
0927 22:58:13.393762 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:13.393801 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:13.394077 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:13.394607 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:13.394611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:13.394636 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:13.395141 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.395150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.396877 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.396883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.399366 (+ 2483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.399383 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.401349 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.401357 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.401553 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.401587 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 22:58:13.401725 (+ 138us) client_negotiation.cc:240] Negotiation successful
0927 22:58:13.401897 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":37,"thread_start_us":596,"threads_started":1}
I20260927 22:58:13.402828 12183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba5a5ae5dbef4e1ca409d2b5630302b9" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58eaf343a3494f928b654160912dc70e" is_pre_election: true
I20260927 22:58:13.403148 12183 raft_consensus.cc:2454] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 0.
I20260927 22:58:13.403481 12468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba5a5ae5dbef4e1ca409d2b5630302b9" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25527d7eed742e2a74c367feb69fcac" is_pre_election: true
I20260927 22:58:13.403764 12256 leader_election.cc:302] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58eaf343a3494f928b654160912dc70e, 5a92d5e4ce1c49b6a08184b000c84107; no voters:
I20260927 22:58:13.404111 12468 raft_consensus.cc:2454] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 0.
I20260927 22:58:13.404248 12543 raft_consensus.cc:2781] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:58:13.404399 12543 raft_consensus.cc:482] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:58:13.404490 12543 raft_consensus.cc:3037] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:13.407799 12543 raft_consensus.cc:504] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.408584 12543 leader_election.cc:288] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 election: Requested vote from peers 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389), d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877)
I20260927 22:58:13.409283 12468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba5a5ae5dbef4e1ca409d2b5630302b9" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25527d7eed742e2a74c367feb69fcac"
I20260927 22:58:13.409376 12183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba5a5ae5dbef4e1ca409d2b5630302b9" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58eaf343a3494f928b654160912dc70e"
I20260927 22:58:13.409557 12468 raft_consensus.cc:3037] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:13.409718 12183 raft_consensus.cc:3037] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:13.412822 12183 raft_consensus.cc:2454] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 1.
I20260927 22:58:13.413188 12468 raft_consensus.cc:2454] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 1.
I20260927 22:58:13.413368 12256 leader_election.cc:302] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58eaf343a3494f928b654160912dc70e, 5a92d5e4ce1c49b6a08184b000c84107; no voters:
I20260927 22:58:13.413834 12543 raft_consensus.cc:2781] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:13.414341 12543 raft_consensus.cc:686] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 LEADER]: Becoming Leader. State: Replica: 5a92d5e4ce1c49b6a08184b000c84107, State: Running, Role: LEADER
I20260927 22:58:13.414810 12543 consensus_queue.cc:237] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.419229 12024 catalog_manager.cc:5495] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } health_report { overall_health: UNKNOWN } } }
Sep 27 22:58:13 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549890, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.190@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:13.499725 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.482554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35629 (local address 127.0.0.1:33404)
0927 22:58:13.483340 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:13.483372 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:13.483602 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:13.484001 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:13.484006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:13.484928 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:13.485212 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.485222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.486502 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.486507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.487535 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.487600 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.488343 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.488349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.488543 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.489314 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:13.489333 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:13.491831 (+ 2498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:13.496539 (+ 4708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:13.496553 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:13.496560 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:13.497069 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:13.497610 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:13.497614 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:13.497617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:13.497789 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:13.498516 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:13.498525 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:13.498756 (+ 231us) client_negotiation.cc:769] Sending connection context
0927 22:58:13.498937 (+ 181us) client_negotiation.cc:240] Negotiation successful
0927 22:58:13.499239 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":419,"threads_started":1}
I20260927 22:58:13.500041 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.482651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.4.74.190:35629)
0927 22:58:13.483713 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.483721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.483750 (+ 29us) server_negotiation.cc:407] Connection header received
0927 22:58:13.483791 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.483794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.483853 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.483973 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:13.485397 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.486356 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.487728 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.488176 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.488321 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.492253 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:13.492263 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:13.492270 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:13.492333 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:13.496280 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:13.497250 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:13.497256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:13.497261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:13.497435 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:13.497934 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:13.497939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:13.497943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:13.498336 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:13.498501 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:13.499099 (+ 598us) server_negotiation.cc:299] Negotiation successful
0927 22:58:13.499449 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":915,"thread_start_us":718,"threads_started":1}
I20260927 22:58:13.506878 12024 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:33404:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 22:58:13.508759 12024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:58:13.517757 12163 tablet_service.cc:1467] Processing CreateTablet for tablet 3128bf893ec7485ebf92b440892962e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=23f082f3e0f34e9eb4a2d50ee801c964]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 22:58:13.518446 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3128bf893ec7485ebf92b440892962e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:13.518146 12303 tablet_service.cc:1467] Processing CreateTablet for tablet 3128bf893ec7485ebf92b440892962e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=23f082f3e0f34e9eb4a2d50ee801c964]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 22:58:13.519018 12303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3128bf893ec7485ebf92b440892962e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:13.522409 12443 tablet_service.cc:1467] Processing CreateTablet for tablet 3128bf893ec7485ebf92b440892962e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=23f082f3e0f34e9eb4a2d50ee801c964]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 22:58:13.523152 12443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3128bf893ec7485ebf92b440892962e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:13.528350 12537 tablet_bootstrap.cc:492] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: Bootstrap starting.
I20260927 22:58:13.530359 12537 tablet_bootstrap.cc:654] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:13.539458 12539 tablet_bootstrap.cc:492] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: Bootstrap starting.
I20260927 22:58:13.539816 12538 tablet_bootstrap.cc:492] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: Bootstrap starting.
I20260927 22:58:13.541621 12537 tablet_bootstrap.cc:492] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: No bootstrap required, opened a new log
I20260927 22:58:13.541772 12538 tablet_bootstrap.cc:654] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:13.541878 12539 tablet_bootstrap.cc:654] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:13.542068 12537 ts_tablet_manager.cc:1397] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.009s
I20260927 22:58:13.543087 12537 raft_consensus.cc:348] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.543503 12537 raft_consensus.cc:374] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:13.543665 12537 raft_consensus.cc:729] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a92d5e4ce1c49b6a08184b000c84107, State: Initialized, Role: FOLLOWER
I20260927 22:58:13.544160 12537 consensus_queue.cc:260] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.544952 12543 tablet_replica.cc:439] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } }
I20260927 22:58:13.545372 12543 tablet_replica.cc:442] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:58:13.546715 12537 ts_tablet_manager.cc:1428] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260927 22:58:13.558573 12539 tablet_bootstrap.cc:492] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: No bootstrap required, opened a new log
I20260927 22:58:13.558938 12539 ts_tablet_manager.cc:1397] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.001s
I20260927 22:58:13.559695 12538 tablet_bootstrap.cc:492] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: No bootstrap required, opened a new log
I20260927 22:58:13.559904 12538 ts_tablet_manager.cc:1397] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.004s
I20260927 22:58:13.560189 12539 raft_consensus.cc:348] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.560599 12539 raft_consensus.cc:374] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:13.560762 12539 raft_consensus.cc:729] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25527d7eed742e2a74c367feb69fcac, State: Initialized, Role: FOLLOWER
I20260927 22:58:13.560819 12538 raft_consensus.cc:348] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.561093 12538 raft_consensus.cc:374] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:13.561185 12538 raft_consensus.cc:729] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58eaf343a3494f928b654160912dc70e, State: Initialized, Role: FOLLOWER
I20260927 22:58:13.561309 12539 consensus_queue.cc:260] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.561517 12538 consensus_queue.cc:260] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.562018 12559 tablet_replica.cc:439] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } }
I20260927 22:58:13.562284 12538 ts_tablet_manager.cc:1428] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 22:58:13.562076 12545 tablet_replica.cc:439] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } }
I20260927 22:58:13.562362 12559 tablet_replica.cc:442] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:58:13.562457 12545 tablet_replica.cc:442] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:58:13.564749 12539 ts_tablet_manager.cc:1428] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260927 22:58:13.582439 12543 raft_consensus.cc:482] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:58:13.582748 12543 raft_consensus.cc:504] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.583727 12543 leader_election.cc:288] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389), d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877)
I20260927 22:58:13.584446 12183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128bf893ec7485ebf92b440892962e4" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58eaf343a3494f928b654160912dc70e" is_pre_election: true
I20260927 22:58:13.584509 12468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128bf893ec7485ebf92b440892962e4" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25527d7eed742e2a74c367feb69fcac" is_pre_election: true
I20260927 22:58:13.584784 12183 raft_consensus.cc:2454] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 0.
I20260927 22:58:13.584832 12468 raft_consensus.cc:2454] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 0.
I20260927 22:58:13.585321 12256 leader_election.cc:302] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58eaf343a3494f928b654160912dc70e, 5a92d5e4ce1c49b6a08184b000c84107; no voters:
I20260927 22:58:13.585660 12543 raft_consensus.cc:2781] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:58:13.585760 12543 raft_consensus.cc:482] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:58:13.585883 12543 raft_consensus.cc:3037] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:13.588274 12543 raft_consensus.cc:504] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.588958 12543 leader_election.cc:288] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 election: Requested vote from peers 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389), d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877)
I20260927 22:58:13.589391 12183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128bf893ec7485ebf92b440892962e4" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58eaf343a3494f928b654160912dc70e"
I20260927 22:58:13.589690 12183 raft_consensus.cc:3037] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:13.589625 12468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3128bf893ec7485ebf92b440892962e4" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25527d7eed742e2a74c367feb69fcac"
I20260927 22:58:13.589902 12468 raft_consensus.cc:3037] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:13.592319 12183 raft_consensus.cc:2454] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 1.
I20260927 22:58:13.593246 12256 leader_election.cc:302] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58eaf343a3494f928b654160912dc70e, 5a92d5e4ce1c49b6a08184b000c84107; no voters:
I20260927 22:58:13.593573 12468 raft_consensus.cc:2454] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 1.
I20260927 22:58:13.593652 12543 raft_consensus.cc:2781] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:13.593863 12543 raft_consensus.cc:686] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 LEADER]: Becoming Leader. State: Replica: 5a92d5e4ce1c49b6a08184b000c84107, State: Running, Role: LEADER
I20260927 22:58:13.594236 12543 consensus_queue.cc:237] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.597337 12570 tablet_replica.cc:439] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: TxnStatusTablet state changed. Reason: New leader 5a92d5e4ce1c49b6a08184b000c84107. Latest consensus state: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } }
I20260927 22:58:13.597862 12570 tablet_replica.cc:442] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: This TxnStatusTablet replica's current role is: LEADER
I20260927 22:58:13.599735 12583 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 22:58:13.600070 12583 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 22:58:13.600263 12025 catalog_manager.cc:5495] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } health_report { overall_health: UNKNOWN } } }
I20260927 22:58:13.646814 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.633816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:42859 (local address 127.0.0.1:58232)
0927 22:58:13.634418 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:13.634441 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:13.634616 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:13.639151 (+ 4535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:13.639156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:13.639173 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 22:58:13.639419 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.639431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.641357 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.641363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.642981 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.642999 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.644180 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.644187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.644437 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.644527 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 22:58:13.646005 (+ 1478us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 22:58:13.646030 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 22:58:13.646133 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 22:58:13.646395 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374}
I20260927 22:58:13.646847 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.634917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58232 (local address 127.4.74.130:42859)
0927 22:58:13.638448 (+ 3531us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.638461 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.638500 (+ 39us) server_negotiation.cc:407] Connection header received
0927 22:58:13.638656 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.638663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.638798 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.639072 (+ 274us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:58:13.639708 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.641146 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.643199 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.643965 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.644146 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.644822 (+ 676us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:58:13.645855 (+ 1033us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:58:13.646020 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:13.646205 (+ 185us) server_negotiation.cc:299] Negotiation successful
0927 22:58:13.646313 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3328,"thread_start_us":3041,"threads_started":1}
I20260927 22:58:13.652812 12468 raft_consensus.cc:1261] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Refusing update from remote peer 5a92d5e4ce1c49b6a08184b000c84107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:13.652923 12183 raft_consensus.cc:1261] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Refusing update from remote peer 5a92d5e4ce1c49b6a08184b000c84107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:13.653779 12543 consensus_queue.cc:1035] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:13.654276 12570 consensus_queue.cc:1035] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:13.677030 12559 tablet_replica.cc:439] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: TxnStatusTablet state changed. Reason: New leader 5a92d5e4ce1c49b6a08184b000c84107. Latest consensus state: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } }
I20260927 22:58:13.677629 12559 tablet_replica.cc:442] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:58:13.679682 12554 mvcc.cc:205] Tried to move back new op lower bound from 7334092364392009728 to 7334092364169830400. Current Snapshot: MvccSnapshot[applied={T|T < 7334092364392009728}]
I20260927 22:58:13.682852 12559 tablet_replica.cc:439] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } }
I20260927 22:58:13.683136 12559 tablet_replica.cc:442] T 3128bf893ec7485ebf92b440892962e4 P d25527d7eed742e2a74c367feb69fcac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:58:13.686334 12543 tablet_replica.cc:439] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } }
I20260927 22:58:13.687009 12543 tablet_replica.cc:442] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: This TxnStatusTablet replica's current role is: LEADER
I20260927 22:58:13.695366 12545 tablet_replica.cc:439] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: TxnStatusTablet state changed. Reason: New leader 5a92d5e4ce1c49b6a08184b000c84107. Latest consensus state: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } }
I20260927 22:58:13.695715 12545 tablet_replica.cc:442] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:58:13.698836 12543 tablet_replica.cc:439] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } }
I20260927 22:58:13.699129 12543 tablet_replica.cc:442] T 3128bf893ec7485ebf92b440892962e4 P 5a92d5e4ce1c49b6a08184b000c84107: This TxnStatusTablet replica's current role is: LEADER
I20260927 22:58:13.701109 12587 mvcc.cc:205] Tried to move back new op lower bound from 7334092364392009728 to 7334092364169830400. Current Snapshot: MvccSnapshot[applied={T|T < 7334092364392009728}]
I20260927 22:58:13.706000 12545 tablet_replica.cc:439] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } } }
I20260927 22:58:13.706373 12545 tablet_replica.cc:442] T 3128bf893ec7485ebf92b440892962e4 P 58eaf343a3494f928b654160912dc70e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 22:58:13.723315 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.713096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.4.74.130:42859)
0927 22:58:13.713285 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.713291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.716288 (+ 2997us) server_negotiation.cc:407] Connection header received
0927 22:58:13.716507 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.716513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.716633 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.716750 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:58:13.717236 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.718597 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.720736 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.721634 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.721776 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.722280 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:58:13.722444 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:58:13.722552 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:13.722808 (+ 256us) server_negotiation.cc:299] Negotiation successful
0927 22:58:13.722910 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 22:58:13.745718 12598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.738421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:42859 (local address 127.4.74.130:40665)
0927 22:58:13.739216 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:13.739233 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:13.739371 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:13.739915 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:13.739919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:13.739930 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 22:58:13.740317 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.740325 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.741915 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.741919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.743219 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:13.743231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.744347 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.744353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:13.744497 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.744559 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 22:58:13.745020 (+ 461us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 22:58:13.745059 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 22:58:13.745166 (+ 107us) client_negotiation.cc:240] Negotiation successful
0927 22:58:13.745276 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":444,"threads_started":1}
I20260927 22:58:13.745811 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.738367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:40665 (local address 127.4.74.130:42859)
0927 22:58:13.738609 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.738617 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.739338 (+ 721us) server_negotiation.cc:407] Connection header received
0927 22:58:13.739531 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.739537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.739745 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.739857 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:58:13.740481 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.741711 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.743373 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.744130 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.744276 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.744774 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:58:13.744896 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:58:13.745021 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:13.745239 (+ 218us) server_negotiation.cc:299] Negotiation successful
0927 22:58:13.745367 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":71}
I20260927 22:58:13.757174 12466 raft_consensus.cc:1261] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Refusing update from remote peer 5a92d5e4ce1c49b6a08184b000c84107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:13.757881 12182 raft_consensus.cc:1261] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Refusing update from remote peer 5a92d5e4ce1c49b6a08184b000c84107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:13.757818 12543 consensus_queue.cc:1035] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:13.758692 12543 consensus_queue.cc:1035] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:13.831808 12025 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:13.832033 12025 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:13.836014 12025 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570: Sending DeleteTablet for 3 replicas of tablet ba5a5ae5dbef4e1ca409d2b5630302b9
I20260927 22:58:13.837206 12303 tablet_service.cc:1514] Processing DeleteTablet for tablet ba5a5ae5dbef4e1ca409d2b5630302b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:13 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:58216
I20260927 22:58:13.837200 12163 tablet_service.cc:1514] Processing DeleteTablet for tablet ba5a5ae5dbef4e1ca409d2b5630302b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:13 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:39986
I20260927 22:58:13.837683 12443 tablet_service.cc:1514] Processing DeleteTablet for tablet ba5a5ae5dbef4e1ca409d2b5630302b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:13 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:48102
I20260927 22:58:13.838272 12606 tablet_replica.cc:330] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e: stopping tablet replica
I20260927 22:58:13.838606 12606 raft_consensus.cc:2229] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:13.839051 12606 raft_consensus.cc:2258] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:13.840821 12606 ts_tablet_manager.cc:1905] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:13.849817 12607 tablet_replica.cc:330] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac: stopping tablet replica
I20260927 22:58:13.850397 12608 tablet_replica.cc:330] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107: stopping tablet replica
I20260927 22:58:13.850735 12607 raft_consensus.cc:2229] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:13.851064 12608 raft_consensus.cc:2229] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:58:13.851379 12606 ts_tablet_manager.cc:1918] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:58:13.851437 12607 raft_consensus.cc:2258] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:13.851737 12606 log.cc:1196] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/ba5a5ae5dbef4e1ca409d2b5630302b9
I20260927 22:58:13.852210 12608 raft_consensus.cc:2258] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:13.852406 12606 ts_tablet_manager.cc:1939] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 58eaf343a3494f928b654160912dc70e: Deleting consensus metadata
I20260927 22:58:13.853304 12607 ts_tablet_manager.cc:1905] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:13.853783 12013 catalog_manager.cc:4841] TS 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389): tablet ba5a5ae5dbef4e1ca409d2b5630302b9 (table test-table [id=7d5420fda4174beeb31db2d040eddafe]) successfully deleted
I20260927 22:58:13.855083 12608 ts_tablet_manager.cc:1905] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:13.859051 12607 ts_tablet_manager.cc:1918] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:58:13.859211 12607 log.cc:1196] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/ba5a5ae5dbef4e1ca409d2b5630302b9
I20260927 22:58:13.859728 12607 ts_tablet_manager.cc:1939] T ba5a5ae5dbef4e1ca409d2b5630302b9 P d25527d7eed742e2a74c367feb69fcac: Deleting consensus metadata
I20260927 22:58:13.861025 12011 catalog_manager.cc:4841] TS d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877): tablet ba5a5ae5dbef4e1ca409d2b5630302b9 (table test-table [id=7d5420fda4174beeb31db2d040eddafe]) successfully deleted
I20260927 22:58:13.864276 12608 ts_tablet_manager.cc:1918] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:58:13.864470 12608 log.cc:1196] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/ba5a5ae5dbef4e1ca409d2b5630302b9
I20260927 22:58:13.865006 12608 ts_tablet_manager.cc:1939] T ba5a5ae5dbef4e1ca409d2b5630302b9 P 5a92d5e4ce1c49b6a08184b000c84107: Deleting consensus metadata
I20260927 22:58:13.866500 12010 catalog_manager.cc:4841] TS 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130:42859): tablet ba5a5ae5dbef4e1ca409d2b5630302b9 (table test-table [id=7d5420fda4174beeb31db2d040eddafe]) successfully deleted
Sep 27 22:58:13 dist-test-slave-5pd3 krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 22:58:13 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:13 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 22:58:13.904932 12578 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:13.893496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33418 (local address 127.4.74.190:35629)
0927 22:58:13.893786 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.893794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.894165 (+ 371us) server_negotiation.cc:407] Connection header received
0927 22:58:13.894452 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.894457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.894540 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.894650 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:13.896066 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.896839 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.898088 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.898515 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.898651 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.904653 (+ 6002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33418: BlockingRecv error: recv got EOF from 127.0.0.1:33418 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 27 22:58:13 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM
I20260927 22:58:13.927246 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:13.908292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33422 (local address 127.4.74.190:35629)
0927 22:58:13.908461 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:13.908465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:13.908940 (+ 475us) server_negotiation.cc:407] Connection header received
0927 22:58:13.909168 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:13.909172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:13.909237 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:13.909343 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:13.911323 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.912298 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.914270 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:13.914741 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:13.914855 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:13.920518 (+ 5663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:13.920523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:13.920526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:13.920569 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:13.924135 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:13.924958 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:13.924963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:13.924968 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:13.925107 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:13.925503 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:13.925510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:13.925514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:13.925793 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:13.926060 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:13.926713 (+ 653us) server_negotiation.cc:299] Negotiation successful
0927 22:58:13.926956 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 22:58:13.932201 12025 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:58:13.933007 12025 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:58:13.940454 12163 tablet_service.cc:1467] Processing CreateTablet for tablet 8cab9ade9bef465f8934c02679580b71 (DEFAULT_TABLE table=test-table [id=9db6612781a74f70a1869b72a1efbf9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:13.941026 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cab9ade9bef465f8934c02679580b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:13.944253 12443 tablet_service.cc:1467] Processing CreateTablet for tablet 8cab9ade9bef465f8934c02679580b71 (DEFAULT_TABLE table=test-table [id=9db6612781a74f70a1869b72a1efbf9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:13.944620 12443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cab9ade9bef465f8934c02679580b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:13.949834 12539 tablet_bootstrap.cc:492] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: Bootstrap starting.
I20260927 22:58:13.950558 12303 tablet_service.cc:1467] Processing CreateTablet for tablet 8cab9ade9bef465f8934c02679580b71 (DEFAULT_TABLE table=test-table [id=9db6612781a74f70a1869b72a1efbf9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:13.951030 12303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cab9ade9bef465f8934c02679580b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:13.951610 12539 tablet_bootstrap.cc:654] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:13.956853 12538 tablet_bootstrap.cc:492] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e: Bootstrap starting.
I20260927 22:58:13.958760 12538 tablet_bootstrap.cc:654] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:13.960896 12537 tablet_bootstrap.cc:492] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107: Bootstrap starting.
I20260927 22:58:13.963090 12537 tablet_bootstrap.cc:654] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:13.965662 12539 tablet_bootstrap.cc:492] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: No bootstrap required, opened a new log
I20260927 22:58:13.965867 12539 ts_tablet_manager.cc:1397] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.010s
I20260927 22:58:13.967130 12539 raft_consensus.cc:348] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.967404 12539 raft_consensus.cc:374] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:13.967514 12539 raft_consensus.cc:729] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25527d7eed742e2a74c367feb69fcac, State: Initialized, Role: FOLLOWER
I20260927 22:58:13.967837 12539 consensus_queue.cc:260] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.971786 12538 tablet_bootstrap.cc:492] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e: No bootstrap required, opened a new log
I20260927 22:58:13.971880 12539 ts_tablet_manager.cc:1428] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260927 22:58:13.971987 12538 ts_tablet_manager.cc:1397] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.005s
I20260927 22:58:13.972795 12538 raft_consensus.cc:348] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.973074 12538 raft_consensus.cc:374] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:13.973173 12538 raft_consensus.cc:729] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58eaf343a3494f928b654160912dc70e, State: Initialized, Role: FOLLOWER
I20260927 22:58:13.973410 12538 consensus_queue.cc:260] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.974097 12538 ts_tablet_manager.cc:1428] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 22:58:13.977953 12537 tablet_bootstrap.cc:492] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107: No bootstrap required, opened a new log
I20260927 22:58:13.978217 12537 ts_tablet_manager.cc:1397] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.008s
I20260927 22:58:13.979331 12537 raft_consensus.cc:348] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.979682 12537 raft_consensus.cc:374] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:13.979813 12537 raft_consensus.cc:729] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a92d5e4ce1c49b6a08184b000c84107, State: Initialized, Role: FOLLOWER
I20260927 22:58:13.980083 12537 consensus_queue.cc:260] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:13.980701 12537 ts_tablet_manager.cc:1428] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 22:58:14.246186 12570 raft_consensus.cc:482] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:58:14.246477 12570 raft_consensus.cc:504] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.247336 12570 leader_election.cc:288] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389), d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877)
I20260927 22:58:14.248039 12468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cab9ade9bef465f8934c02679580b71" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25527d7eed742e2a74c367feb69fcac" is_pre_election: true
I20260927 22:58:14.248014 12182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cab9ade9bef465f8934c02679580b71" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58eaf343a3494f928b654160912dc70e" is_pre_election: true
I20260927 22:58:14.248296 12468 raft_consensus.cc:2454] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 0.
I20260927 22:58:14.248395 12182 raft_consensus.cc:2454] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 0.
I20260927 22:58:14.248790 12254 leader_election.cc:302] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a92d5e4ce1c49b6a08184b000c84107, d25527d7eed742e2a74c367feb69fcac; no voters:
I20260927 22:58:14.249132 12570 raft_consensus.cc:2781] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:58:14.249251 12570 raft_consensus.cc:482] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:58:14.249429 12570 raft_consensus.cc:3037] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:14.252004 12570 raft_consensus.cc:504] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.252689 12570 leader_election.cc:288] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 election: Requested vote from peers 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389), d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877)
I20260927 22:58:14.253288 12468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cab9ade9bef465f8934c02679580b71" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25527d7eed742e2a74c367feb69fcac"
I20260927 22:58:14.253264 12182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cab9ade9bef465f8934c02679580b71" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58eaf343a3494f928b654160912dc70e"
I20260927 22:58:14.253547 12468 raft_consensus.cc:3037] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:14.253580 12182 raft_consensus.cc:3037] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:14.255641 12468 raft_consensus.cc:2454] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 1.
I20260927 22:58:14.256141 12254 leader_election.cc:302] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a92d5e4ce1c49b6a08184b000c84107, d25527d7eed742e2a74c367feb69fcac; no voters:
I20260927 22:58:14.256397 12570 raft_consensus.cc:2781] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:14.256598 12570 raft_consensus.cc:686] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 LEADER]: Becoming Leader. State: Replica: 5a92d5e4ce1c49b6a08184b000c84107, State: Running, Role: LEADER
I20260927 22:58:14.257045 12182 raft_consensus.cc:2454] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 1.
I20260927 22:58:14.256992 12570 consensus_queue.cc:237] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.260040 12025 catalog_manager.cc:5495] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } health_report { overall_health: UNKNOWN } } }
I20260927 22:58:14.328087 12626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:14.321164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58260 (local address 127.4.74.130:42859)
0927 22:58:14.321733 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:14.321740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:14.321837 (+ 97us) server_negotiation.cc:407] Connection header received
0927 22:58:14.322067 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:14.322072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:14.322187 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:14.322330 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:58:14.322907 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:14.323738 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:14.325829 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:14.326385 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:14.326507 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:14.327150 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:58:14.327263 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:58:14.327404 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:14.327632 (+ 228us) server_negotiation.cc:299] Negotiation successful
0927 22:58:14.327718 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":318,"threads_started":1}
I20260927 22:58:14.331461 12468 raft_consensus.cc:1261] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Refusing update from remote peer 5a92d5e4ce1c49b6a08184b000c84107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:14.331441 12182 raft_consensus.cc:1261] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Refusing update from remote peer 5a92d5e4ce1c49b6a08184b000c84107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:14.332160 12570 consensus_queue.cc:1035] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:14.332476 12601 consensus_queue.cc:1035] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:14.348254 12025 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:14.348611 12025 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:14.352499 12025 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570: Sending DeleteTablet for 3 replicas of tablet 8cab9ade9bef465f8934c02679580b71
I20260927 22:58:14.354475 12303 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cab9ade9bef465f8934c02679580b71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:14 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:58216
I20260927 22:58:14.354710 12443 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cab9ade9bef465f8934c02679580b71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:14 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:48102
I20260927 22:58:14.355023 12607 tablet_replica.cc:330] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: stopping tablet replica
I20260927 22:58:14.354955 12163 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cab9ade9bef465f8934c02679580b71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:14 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:39986
I20260927 22:58:14.355432 12608 tablet_replica.cc:330] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107: stopping tablet replica
I20260927 22:58:14.355703 12608 raft_consensus.cc:2229] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:58:14.356019 12608 raft_consensus.cc:2258] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:14.356213 12024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:58:14.357014 12024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:58:14.358062 12608 ts_tablet_manager.cc:1905] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:14.364959 12633 tablet_replica.cc:330] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e: stopping tablet replica
I20260927 22:58:14.365768 12443 tablet_service.cc:1467] Processing CreateTablet for tablet 54ef33e7400349ae81b985ec72b7bfdc (DEFAULT_TABLE table=test-table [id=ba8c8e4138bd4a6d8a81ed2eb6965ace]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:14.366307 12443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ef33e7400349ae81b985ec72b7bfdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:14.368813 12163 tablet_service.cc:1467] Processing CreateTablet for tablet 54ef33e7400349ae81b985ec72b7bfdc (DEFAULT_TABLE table=test-table [id=ba8c8e4138bd4a6d8a81ed2eb6965ace]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:14.369375 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ef33e7400349ae81b985ec72b7bfdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:14.375993 12303 tablet_service.cc:1467] Processing CreateTablet for tablet 54ef33e7400349ae81b985ec72b7bfdc (DEFAULT_TABLE table=test-table [id=ba8c8e4138bd4a6d8a81ed2eb6965ace]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:14.377077 12303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54ef33e7400349ae81b985ec72b7bfdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:14.379395 12539 tablet_bootstrap.cc:492] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac: Bootstrap starting.
I20260927 22:58:14.381281 12539 tablet_bootstrap.cc:654] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:14.384613 12539 tablet_bootstrap.cc:492] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac: No bootstrap required, opened a new log
I20260927 22:58:14.384785 12539 ts_tablet_manager.cc:1397] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260927 22:58:14.385952 12539 raft_consensus.cc:348] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.386247 12539 raft_consensus.cc:374] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:14.386353 12539 raft_consensus.cc:729] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25527d7eed742e2a74c367feb69fcac, State: Initialized, Role: FOLLOWER
I20260927 22:58:14.387013 12539 consensus_queue.cc:260] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.387598 12539 ts_tablet_manager.cc:1428] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 22:58:14.388223 12538 tablet_bootstrap.cc:492] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e: Bootstrap starting.
I20260927 22:58:14.388867 12608 ts_tablet_manager.cc:1918] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:14.389070 12608 log.cc:1196] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/8cab9ade9bef465f8934c02679580b71
I20260927 22:58:14.389669 12608 ts_tablet_manager.cc:1939] T 8cab9ade9bef465f8934c02679580b71 P 5a92d5e4ce1c49b6a08184b000c84107: Deleting consensus metadata
I20260927 22:58:14.390340 12538 tablet_bootstrap.cc:654] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:14.391085 12537 tablet_bootstrap.cc:492] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107: Bootstrap starting.
I20260927 22:58:14.391150 12010 catalog_manager.cc:4841] TS 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130:42859): tablet 8cab9ade9bef465f8934c02679580b71 (table test-table [id=9db6612781a74f70a1869b72a1efbf9e]) successfully deleted
I20260927 22:58:14.393399 12537 tablet_bootstrap.cc:654] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:14.394351 12538 tablet_bootstrap.cc:492] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e: No bootstrap required, opened a new log
I20260927 22:58:14.394568 12538 ts_tablet_manager.cc:1397] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260927 22:58:14.395655 12538 raft_consensus.cc:348] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.396071 12538 raft_consensus.cc:374] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:14.396173 12538 raft_consensus.cc:729] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58eaf343a3494f928b654160912dc70e, State: Initialized, Role: FOLLOWER
I20260927 22:58:14.396579 12538 consensus_queue.cc:260] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.397495 12538 ts_tablet_manager.cc:1428] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 22:58:14.398836 12633 raft_consensus.cc:2229] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:14.399686 12633 raft_consensus.cc:2258] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:14.401083 12633 ts_tablet_manager.cc:1905] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:14.406551 12537 tablet_bootstrap.cc:492] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107: No bootstrap required, opened a new log
I20260927 22:58:14.406865 12537 ts_tablet_manager.cc:1397] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.001s
I20260927 22:58:14.408565 12633 ts_tablet_manager.cc:1918] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:14.408439 12537 raft_consensus.cc:348] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.408733 12633 log.cc:1196] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/8cab9ade9bef465f8934c02679580b71
I20260927 22:58:14.408779 12537 raft_consensus.cc:374] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:14.408875 12537 raft_consensus.cc:729] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a92d5e4ce1c49b6a08184b000c84107, State: Initialized, Role: FOLLOWER
I20260927 22:58:14.409293 12633 ts_tablet_manager.cc:1939] T 8cab9ade9bef465f8934c02679580b71 P 58eaf343a3494f928b654160912dc70e: Deleting consensus metadata
I20260927 22:58:14.409308 12537 consensus_queue.cc:260] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.411051 12013 catalog_manager.cc:4841] TS 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389): tablet 8cab9ade9bef465f8934c02679580b71 (table test-table [id=9db6612781a74f70a1869b72a1efbf9e]) successfully deleted
I20260927 22:58:14.413440 12537 ts_tablet_manager.cc:1428] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260927 22:58:14.414758 12607 raft_consensus.cc:2229] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:14.415309 12607 pending_rounds.cc:62] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: Trying to abort 2 pending ops.
I20260927 22:58:14.415474 12607 pending_rounds.cc:68] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7334092367174635520 op_type: WRITE_OP write_request { tablet_id: "8cab9ade9bef465f8934c02679580b71" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\262\301\346\325\006\"8\n\ntest-admin\022*\n 9db6612781a74f70a1869b72a1efbf9e\020\001\030\001 \001(\001" signature: "<redacted>""O\375\311-\022\231\261\210\005\237\345\t\342\244>\222Z\256\367\237\227BX\247\352\372\307W\3251[\231!\230\245V\266\376}V\nFmN\276\247\004\332\356l%\310\257#\010\327\014=\367z\356C\250\243" signing_key_seq_num: 0 } } request_id { client_id: "5c6865d4c12947589f9d1d98da79806f" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260927 22:58:14.416318 12607 pending_rounds.cc:68] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334092366879309824 op_type: NO_OP noop_request { }
I20260927 22:58:14.416486 12607 raft_consensus.cc:2866] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260927 22:58:14.416637 12607 raft_consensus.cc:2258] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 22:58:14.418095 12637 consensus_queue.cc:1159] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d25527d7eed742e2a74c367feb69fcac" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260927 22:58:14.419211 12607 ts_tablet_manager.cc:1905] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:14.425329 12607 ts_tablet_manager.cc:1918] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:14.425552 12607 log.cc:1196] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/8cab9ade9bef465f8934c02679580b71
I20260927 22:58:14.425947 12607 ts_tablet_manager.cc:1939] T 8cab9ade9bef465f8934c02679580b71 P d25527d7eed742e2a74c367feb69fcac: Deleting consensus metadata
I20260927 22:58:14.427191 12011 catalog_manager.cc:4841] TS d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877): tablet 8cab9ade9bef465f8934c02679580b71 (table test-table [id=9db6612781a74f70a1869b72a1efbf9e]) successfully deleted
I20260927 22:58:14.448020 12601 raft_consensus.cc:482] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:58:14.448380 12601 raft_consensus.cc:504] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.449252 12601 leader_election.cc:288] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389), d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877)
I20260927 22:58:14.450088 12468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ef33e7400349ae81b985ec72b7bfdc" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25527d7eed742e2a74c367feb69fcac" is_pre_election: true
I20260927 22:58:14.450129 12182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ef33e7400349ae81b985ec72b7bfdc" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58eaf343a3494f928b654160912dc70e" is_pre_election: true
I20260927 22:58:14.450433 12468 raft_consensus.cc:2454] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 0.
I20260927 22:58:14.450664 12182 raft_consensus.cc:2454] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 0.
I20260927 22:58:14.451329 12256 leader_election.cc:302] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 58eaf343a3494f928b654160912dc70e, 5a92d5e4ce1c49b6a08184b000c84107, d25527d7eed742e2a74c367feb69fcac; no voters:
I20260927 22:58:14.451804 12601 raft_consensus.cc:2781] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:58:14.451946 12601 raft_consensus.cc:482] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:58:14.452112 12601 raft_consensus.cc:3037] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:14.454677 12601 raft_consensus.cc:504] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.455437 12601 leader_election.cc:288] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 election: Requested vote from peers 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389), d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877)
I20260927 22:58:14.456009 12182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ef33e7400349ae81b985ec72b7bfdc" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58eaf343a3494f928b654160912dc70e"
I20260927 22:58:14.456099 12468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54ef33e7400349ae81b985ec72b7bfdc" candidate_uuid: "5a92d5e4ce1c49b6a08184b000c84107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25527d7eed742e2a74c367feb69fcac"
I20260927 22:58:14.456292 12182 raft_consensus.cc:3037] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:14.456373 12468 raft_consensus.cc:3037] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:14.458623 12468 raft_consensus.cc:2454] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 1.
I20260927 22:58:14.459259 12254 leader_election.cc:302] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a92d5e4ce1c49b6a08184b000c84107, d25527d7eed742e2a74c367feb69fcac; no voters:
I20260927 22:58:14.459666 12601 raft_consensus.cc:2781] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:14.459650 12182 raft_consensus.cc:2454] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a92d5e4ce1c49b6a08184b000c84107 in term 1.
I20260927 22:58:14.459921 12601 raft_consensus.cc:686] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 LEADER]: Becoming Leader. State: Replica: 5a92d5e4ce1c49b6a08184b000c84107, State: Running, Role: LEADER
I20260927 22:58:14.460289 12601 consensus_queue.cc:237] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } }
I20260927 22:58:14.464061 12025 catalog_manager.cc:5495] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "5a92d5e4ce1c49b6a08184b000c84107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a92d5e4ce1c49b6a08184b000c84107" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 } health_report { overall_health: UNKNOWN } } }
I20260927 22:58:14.511359 12466 raft_consensus.cc:1261] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Refusing update from remote peer 5a92d5e4ce1c49b6a08184b000c84107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:14.511507 12182 raft_consensus.cc:1261] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Refusing update from remote peer 5a92d5e4ce1c49b6a08184b000c84107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:14.512059 12570 consensus_queue.cc:1035] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d25527d7eed742e2a74c367feb69fcac" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 46877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:14.512820 12601 consensus_queue.cc:1035] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58eaf343a3494f928b654160912dc70e" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 43389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:14.595957 12024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:14.596288 12024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:14.600158 12024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28d475a5ed984814881646c88f9ef570: Sending DeleteTablet for 3 replicas of tablet 54ef33e7400349ae81b985ec72b7bfdc
I20260927 22:58:14.601464 12443 tablet_service.cc:1514] Processing DeleteTablet for tablet 54ef33e7400349ae81b985ec72b7bfdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:14 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:48102
I20260927 22:58:14.601502 12163 tablet_service.cc:1514] Processing DeleteTablet for tablet 54ef33e7400349ae81b985ec72b7bfdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:14 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:39986
I20260927 22:58:14.601835 12633 tablet_replica.cc:330] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e: stopping tablet replica
I20260927 22:58:14.601848 12607 tablet_replica.cc:330] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac: stopping tablet replica
I20260927 22:58:14.602061 12607 raft_consensus.cc:2229] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:14.602061 12633 raft_consensus.cc:2229] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:14.602268 12633 raft_consensus.cc:2258] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:14.602322 12607 raft_consensus.cc:2258] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:14.602833 12302 tablet_service.cc:1514] Processing DeleteTablet for tablet 54ef33e7400349ae81b985ec72b7bfdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:14 UTC) from {username='oryx', principal='oryx/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:58216
I20260927 22:58:14.603171 12608 tablet_replica.cc:330] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107: stopping tablet replica
I20260927 22:58:14.603375 12608 raft_consensus.cc:2229] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:58:14.603726 12607 ts_tablet_manager.cc:1905] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:14.603855 12608 raft_consensus.cc:2258] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:14.605363 12608 ts_tablet_manager.cc:1905] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:14.611490 12607 ts_tablet_manager.cc:1918] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:58:14.611701 12607 log.cc:1196] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/54ef33e7400349ae81b985ec72b7bfdc
I20260927 22:58:14.612195 12607 ts_tablet_manager.cc:1939] T 54ef33e7400349ae81b985ec72b7bfdc P d25527d7eed742e2a74c367feb69fcac: Deleting consensus metadata
I20260927 22:58:14.613613 12011 catalog_manager.cc:4841] TS d25527d7eed742e2a74c367feb69fcac (127.4.74.131:46877): tablet 54ef33e7400349ae81b985ec72b7bfdc (table test-table [id=ba8c8e4138bd4a6d8a81ed2eb6965ace]) successfully deleted
I20260927 22:58:14.615037 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 12097
I20260927 22:58:14.616755 12633 ts_tablet_manager.cc:1905] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:14.625316 12608 ts_tablet_manager.cc:1918] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:58:14.625521 12608 log.cc:1196] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/54ef33e7400349ae81b985ec72b7bfdc
I20260927 22:58:14.625983 12608 ts_tablet_manager.cc:1939] T 54ef33e7400349ae81b985ec72b7bfdc P 5a92d5e4ce1c49b6a08184b000c84107: Deleting consensus metadata
I20260927 22:58:14.627367 12010 catalog_manager.cc:4841] TS 5a92d5e4ce1c49b6a08184b000c84107 (127.4.74.130:42859): tablet 54ef33e7400349ae81b985ec72b7bfdc (table test-table [id=ba8c8e4138bd4a6d8a81ed2eb6965ace]) successfully deleted
I20260927 22:58:14.628715 12633 ts_tablet_manager.cc:1918] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 22:58:14.628896 12633 log.cc:1196] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/54ef33e7400349ae81b985ec72b7bfdc
I20260927 22:58:14.629379 12633 ts_tablet_manager.cc:1939] T 54ef33e7400349ae81b985ec72b7bfdc P 58eaf343a3494f928b654160912dc70e: Deleting consensus metadata
I20260927 22:58:14.630645 12013 catalog_manager.cc:4841] TS 58eaf343a3494f928b654160912dc70e (127.4.74.129:43389): tablet 54ef33e7400349ae81b985ec72b7bfdc (table test-table [id=ba8c8e4138bd4a6d8a81ed2eb6965ace]) successfully deleted
Sep 27 22:58:14 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.129@KRBTEST.COM
I20260927 22:58:14.652041 12647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:14.616528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39988 (local address 127.4.74.129:43389)
0927 22:58:14.632087 (+ 15559us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:14.632094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:14.632118 (+ 24us) server_negotiation.cc:407] Connection header received
0927 22:58:14.632185 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:14.632189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:14.632309 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:14.632522 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:14.634748 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:14.635906 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:14.638035 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:14.638699 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:14.638867 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:14.644737 (+ 5870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:14.644759 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:14.644779 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:14.644834 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:14.648491 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:14.649438 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:14.649451 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:14.649454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:14.649657 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:14.650173 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:14.650177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:14.650180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:14.650641 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:14.650898 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:14.651392 (+ 494us) server_negotiation.cc:299] Negotiation successful
0927 22:58:14.651722 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15417,"thread_start_us":15291,"threads_started":1}
I20260927 22:58:14.652957 12228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:14.757248 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 12097
I20260927 22:58:14.776888 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 12237
Sep 27 22:58:14 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.130@KRBTEST.COM
I20260927 22:58:14.794037 12626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:14.777910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58262 (local address 127.4.74.130:42859)
0927 22:58:14.778123 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:14.778128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:14.778150 (+ 22us) server_negotiation.cc:407] Connection header received
0927 22:58:14.778321 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:14.778325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:14.778412 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:14.778528 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:14.780078 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:14.780885 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:14.782288 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:14.782721 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:14.782823 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:14.787745 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:14.787764 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:14.787783 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:14.787824 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:14.790987 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:14.791799 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:14.791806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:14.791809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:14.791950 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:14.792334 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:14.792341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:14.792348 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:14.792749 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:14.793038 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:14.793538 (+ 500us) server_negotiation.cc:299] Negotiation successful
0927 22:58:14.793743 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 22:58:14.794937 12368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:14.947602 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 12237
I20260927 22:58:14.974285 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 12377
Sep 27 22:58:14 dist-test-slave-5pd3 krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.74.131@KRBTEST.COM
I20260927 22:58:14.993393 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:14.975256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48126 (local address 127.4.74.131:46877)
0927 22:58:14.975895 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:14.975900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:14.975916 (+ 16us) server_negotiation.cc:407] Connection header received
0927 22:58:14.975980 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:14.975984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:14.976063 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:14.976199 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:14.977696 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:14.978455 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:14.979971 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:14.980467 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:14.980618 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:14.986180 (+ 5562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:14.986200 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:14.986221 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:14.986277 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:14.989871 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:14.990864 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:14.990876 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:14.990879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:14.991053 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:14.991472 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:14.991484 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:14.991488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:14.992017 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:14.992349 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:14.992710 (+ 361us) server_negotiation.cc:299] Negotiation successful
0927 22:58:14.992983 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":330,"threads_started":1}
I20260927 22:58:14.994311 12508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:15.142467 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 12377
I20260927 22:58:15.168572 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 11994
I20260927 22:58:15.170204 12066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:15.291915 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 11994
2026-09-27T22:58:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6695 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:58:15 dist-test-slave-5pd3 krb5kdc[12655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:58:15 dist-test-slave-5pd3 krb5kdc[12655](info): set up 2 sockets
Sep 27 22:58:15 dist-test-slave-5pd3 krb5kdc[12655](info): commencing operation
krb5kdc: starting...
W20260927 22:58:16.440608 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:58:16 dist-test-slave-5pd3 krb5kdc[12655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:58:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:58:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:58:16.711634 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:38273
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:37907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:38273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:17.051046 12668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:17.051404 12668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:17.051625 12668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:17.059062 12668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:58:17.059224 12668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:17.059337 12668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:17.059434 12668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:58:17.059535 12668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:58:17.069585 12668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:37907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:38273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:38273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:17.070976 12668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:17.072341 12668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:17.084797 12673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:17.085072 12676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:17.085180 12674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:17.086256 12668 server_base.cc:1034] running on GCE node
I20260927 22:58:17.087363 12668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:17.089356 12668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:17.090654 12668 hybrid_clock.cc:648] HybridClock initialized: now 1790549897090564 us; error 71 us; skew 500 ppm
I20260927 22:58:17.091140 12668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-27T22:58:17Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1877 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:58:17 dist-test-slave-5pd3 krb5kdc[12682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:58:17 dist-test-slave-5pd3 krb5kdc[12682](info): set up 2 sockets
Sep 27 22:58:17 dist-test-slave-5pd3 krb5kdc[12682](info): commencing operation
krb5kdc: starting...
W20260927 22:58:18.328639 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:58:18 dist-test-slave-5pd3 krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:58:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:58:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:58:18.597291 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:46117
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:46117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:18.941212 12695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:18.941560 12695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:18.941711 12695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:18.949409 12695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:58:18.949558 12695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:18.949606 12695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:18.949658 12695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:58:18.949704 12695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:58:18.960577 12695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:46117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:46117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:18.962036 12695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:18.963347 12695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:18.977389 12703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:18.977486 12701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:18.978119 12700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:18.978346 12695 server_base.cc:1034] running on GCE node
I20260927 22:58:18.979218 12695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:18.980791 12695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:18.982086 12695 hybrid_clock.cc:648] HybridClock initialized: now 1790549898982005 us; error 58 us; skew 500 ppm
I20260927 22:58:18.982549 12695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:18 dist-test-slave-5pd3 krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549898, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:18.986584 12695 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:58:18.988386 12695 webserver.cc:460] Webserver started at http://127.4.74.190:43385/ using document root <none> and password file <none>
I20260927 22:58:18.989080 12695 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:18.989183 12695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:18.989459 12695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:18.992633 12695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "9411c9826ef740ab9ac23f62f099c5b7"
format_stamp: "Formatted at 2026-09-27 22:58:18 on dist-test-slave-5pd3"
server_key: "fe0d204708ff017984e1cb5eefe12483"
server_key_iv: "135be56265eb8a9eca0d0612e86e16ae"
server_key_version: "encryptionkey@0"
I20260927 22:58:18.993418 12695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "9411c9826ef740ab9ac23f62f099c5b7"
format_stamp: "Formatted at 2026-09-27 22:58:18 on dist-test-slave-5pd3"
server_key: "fe0d204708ff017984e1cb5eefe12483"
server_key_iv: "135be56265eb8a9eca0d0612e86e16ae"
server_key_version: "encryptionkey@0"
I20260927 22:58:18.999207 12695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 22:58:19.003446 12710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:19.005294 12695 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:58:19.005554 12695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "9411c9826ef740ab9ac23f62f099c5b7"
format_stamp: "Formatted at 2026-09-27 22:58:18 on dist-test-slave-5pd3"
server_key: "fe0d204708ff017984e1cb5eefe12483"
server_key_iv: "135be56265eb8a9eca0d0612e86e16ae"
server_key_version: "encryptionkey@0"
I20260927 22:58:19.005781 12695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:19.021222 12695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:19.026181 12695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:19.026533 12695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:19.047690 12695 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:46117
I20260927 22:58:19.047700 12761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:46117 every 8 connection(s)
I20260927 22:58:19.049170 12695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:58:19.054062 12762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:19.057245 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 12695
I20260927 22:58:19.057529 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:58:19.058455 4394 external_mini_cluster.cc:1428] Setting key d4270a6d22d52b53aecbe174c5cb0ea9
I20260927 22:58:19.070230 12762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7: Bootstrap starting.
I20260927 22:58:19.075682 12762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7: Neither blocks nor log segments found. Creating new log.
Sep 27 22:58:19 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:58:19.077054 12762 log.cc:824] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:19.080722 12762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7: No bootstrap required, opened a new log
I20260927 22:58:19.087086 12765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:19.061237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.4.74.190:46117)
0927 22:58:19.062238 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:19.062251 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:19.062297 (+ 46us) server_negotiation.cc:407] Connection header received
0927 22:58:19.064315 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:19.064342 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:19.064814 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:19.065193 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:19.067333 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:19.068817 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:19.070681 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:19.071416 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:19.071632 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:19.077443 (+ 5811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:19.077452 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:19.077471 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:19.077520 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:19.081487 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:19.082582 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:19.082590 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:19.082607 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:19.082866 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:19.083342 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:19.083345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:19.083348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:19.084223 (+ 875us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:19.084512 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:19.085077 (+ 565us) server_negotiation.cc:299] Negotiation successful
0927 22:58:19.085651 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":464,"threads_started":1}
I20260927 22:58:19.090428 12762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } }
I20260927 22:58:19.090905 12762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:19.091042 12762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9411c9826ef740ab9ac23f62f099c5b7, State: Initialized, Role: FOLLOWER
I20260927 22:58:19.091738 12762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } }
I20260927 22:58:19.091958 12762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:58:19.092108 12762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:58:19.092298 12762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:19.094529 12762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } }
I20260927 22:58:19.095070 12762 leader_election.cc:302] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9411c9826ef740ab9ac23f62f099c5b7; no voters:
I20260927 22:58:19.095846 12762 leader_election.cc:288] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:58:19.096052 12767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:19.097462 12767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 1 LEADER]: Becoming Leader. State: Replica: 9411c9826ef740ab9ac23f62f099c5b7, State: Running, Role: LEADER
I20260927 22:58:19.098124 12767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } }
I20260927 22:58:19.098393 12762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:58:19.102452 12768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9411c9826ef740ab9ac23f62f099c5b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } } }
I20260927 22:58:19.102986 12768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:19.103979 12769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9411c9826ef740ab9ac23f62f099c5b7. Latest consensus state: current_term: 1 leader_uuid: "9411c9826ef740ab9ac23f62f099c5b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } } }
I20260927 22:58:19.104451 12769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:19.118255 12774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:58:19.123658 12774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:58:19.135533 12774 catalog_manager.cc:1347] Generated new cluster ID: cfbc168b2f8c478d8568ceaa16861ea2
I20260927 22:58:19.135761 12774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:58:19.150640 12774 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:58:19.152170 12774 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:58:19.161813 12774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7: Generated new TSK 0
I20260927 22:58:19.163200 12774 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:58:19.229007 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46117
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:19.598821 12789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:19.599179 12789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:19.599609 12789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:19.609611 12789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:19.609753 12789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:19.609874 12789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:58:19.620893 12789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:19.622377 12789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:19.623878 12789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:19.641324 12797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:19.641610 12795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:19.641384 12794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:19.642660 12789 server_base.cc:1034] running on GCE node
I20260927 22:58:19.643491 12789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:19.644698 12789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:19.645959 12789 hybrid_clock.cc:648] HybridClock initialized: now 1790549899645902 us; error 51 us; skew 500 ppm
I20260927 22:58:19.646504 12789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:19 dist-test-slave-5pd3 krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549899, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:19.651232 12789 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:58:19.653450 12789 webserver.cc:460] Webserver started at http://127.4.74.129:37793/ using document root <none> and password file <none>
I20260927 22:58:19.654433 12789 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:19.654618 12789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:19.655009 12789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:19.659238 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "c85130c447744c2aab384cb0a88e327d"
format_stamp: "Formatted at 2026-09-27 22:58:19 on dist-test-slave-5pd3"
server_key: "9f63494a75f8d75febd24095d9c2cfc6"
server_key_iv: "2a7e8023789e5e72b84e4d03219a1e53"
server_key_version: "encryptionkey@0"
I20260927 22:58:19.660351 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "c85130c447744c2aab384cb0a88e327d"
format_stamp: "Formatted at 2026-09-27 22:58:19 on dist-test-slave-5pd3"
server_key: "9f63494a75f8d75febd24095d9c2cfc6"
server_key_iv: "2a7e8023789e5e72b84e4d03219a1e53"
server_key_version: "encryptionkey@0"
I20260927 22:58:19.667119 12789 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260927 22:58:19.671993 12804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:19.673784 12789 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:58:19.674052 12789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "c85130c447744c2aab384cb0a88e327d"
format_stamp: "Formatted at 2026-09-27 22:58:19 on dist-test-slave-5pd3"
server_key: "9f63494a75f8d75febd24095d9c2cfc6"
server_key_iv: "2a7e8023789e5e72b84e4d03219a1e53"
server_key_version: "encryptionkey@0"
I20260927 22:58:19.674342 12789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:19.704519 12789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:19.710371 12789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:19.710772 12789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:19.712059 12789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:19.713927 12789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:19.714036 12789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:19.714202 12789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:19.714285 12789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:19.755218 12789 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:42661
I20260927 22:58:19.755272 12916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:42661 every 8 connection(s)
I20260927 22:58:19.756984 12789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:58:19.765122 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 12789
I20260927 22:58:19.765368 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:58:19.766216 4394 external_mini_cluster.cc:1428] Setting key b54963605fd2fd75c1f86abff3e8e5ec
Sep 27 22:58:19 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549899, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:58:19.785526 12919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:19.760623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:44063 (local address 127.4.74.190:46117)
0927 22:58:19.761206 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:19.761211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:19.762726 (+ 1515us) server_negotiation.cc:407] Connection header received
0927 22:58:19.764960 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:19.764966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:19.765064 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:19.765207 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:19.767642 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:19.768570 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:19.771320 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:19.772097 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:19.772324 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:19.777525 (+ 5201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:19.777532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:19.777536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:19.777579 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:19.780929 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:19.782475 (+ 1546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:19.782479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:19.782482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:19.782638 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:19.783296 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:19.783299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:19.783301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:19.783723 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:19.783841 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:19.785046 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0927 22:58:19.785244 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":454,"thread_start_us":298,"threads_started":1}
I20260927 22:58:19.787714 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:19.761432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:44063)
0927 22:58:19.762522 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:19.762574 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:19.764520 (+ 1946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:19.765566 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:19.765580 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:19.766438 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:19.767290 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:19.767316 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:19.768731 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:19.768737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:19.771042 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:19.771067 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:19.772436 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:19.772445 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:19.772787 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:19.773705 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:19.773741 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:19.777106 (+ 3365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:19.781215 (+ 4109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:19.781231 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:19.781260 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:19.782227 (+ 967us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:19.782801 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:19.782810 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:19.782818 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:19.783139 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:19.783892 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:19.783906 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:19.784543 (+ 637us) client_negotiation.cc:769] Sending connection context
0927 22:58:19.784938 (+ 395us) client_negotiation.cc:240] Negotiation successful
0927 22:58:19.785721 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":404,"threads_started":1}
I20260927 22:58:19.789752 12917 heartbeater.cc:347] Connected to a master server at 127.4.74.190:46117
I20260927 22:58:19.790234 12917 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:19.791601 12917 heartbeater.cc:511] Master 127.4.74.190:46117 requested a full tablet report, sending...
I20260927 22:58:19.794486 12727 ts_manager.cc:194] Registered new tserver with Master: c85130c447744c2aab384cb0a88e327d (127.4.74.129:42661)
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 22:58:19.796195 12727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:44063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:58:19.832373 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46117
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:20.241225 12923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:20.241616 12923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:20.241801 12923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:20.249956 12923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:20.250109 12923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:20.250310 12923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:58:20.261548 12923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:20.263266 12923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:20.264871 12923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:20.280901 12929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:20.280944 12928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:20.280871 12931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:20.282213 12923 server_base.cc:1034] running on GCE node
I20260927 22:58:20.282965 12923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:20.284150 12923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:20.285385 12923 hybrid_clock.cc:648] HybridClock initialized: now 1790549900285325 us; error 49 us; skew 500 ppm
I20260927 22:58:20.285770 12923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:20 dist-test-slave-5pd3 krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549900, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:20.289270 12923 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:58:20.290930 12923 webserver.cc:460] Webserver started at http://127.4.74.130:36445/ using document root <none> and password file <none>
I20260927 22:58:20.291716 12923 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:20.291822 12923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:20.292094 12923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:20.295131 12923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "61af181df2a34a9f9233cacf8461e1d4"
format_stamp: "Formatted at 2026-09-27 22:58:20 on dist-test-slave-5pd3"
server_key: "4dc7caa27c6e1e1a5f5c5ec82cf5898e"
server_key_iv: "f6b73b32819a6c52bd286d0c18be7a1e"
server_key_version: "encryptionkey@0"
I20260927 22:58:20.295962 12923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "61af181df2a34a9f9233cacf8461e1d4"
format_stamp: "Formatted at 2026-09-27 22:58:20 on dist-test-slave-5pd3"
server_key: "4dc7caa27c6e1e1a5f5c5ec82cf5898e"
server_key_iv: "f6b73b32819a6c52bd286d0c18be7a1e"
server_key_version: "encryptionkey@0"
I20260927 22:58:20.301695 12923 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 22:58:20.305652 12938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:20.307354 12923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260927 22:58:20.307644 12923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "61af181df2a34a9f9233cacf8461e1d4"
format_stamp: "Formatted at 2026-09-27 22:58:20 on dist-test-slave-5pd3"
server_key: "4dc7caa27c6e1e1a5f5c5ec82cf5898e"
server_key_iv: "f6b73b32819a6c52bd286d0c18be7a1e"
server_key_version: "encryptionkey@0"
I20260927 22:58:20.307879 12923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:20.324785 12923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:20.329233 12923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:20.329574 12923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:20.330734 12923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:20.332345 12923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:20.332465 12923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:20.332631 12923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:20.332713 12923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:20.371356 12923 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:37297
I20260927 22:58:20.371454 13050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:37297 every 8 connection(s)
I20260927 22:58:20.372807 12923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:58:20.381047 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 12923
I20260927 22:58:20.381283 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:58:20.381978 4394 external_mini_cluster.cc:1428] Setting key 67ede08856443430757674e206dfa3a4
Sep 27 22:58:20 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549900, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:58:20.399425 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:20.376094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:41437 (local address 127.4.74.190:46117)
0927 22:58:20.376662 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:20.376670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:20.377903 (+ 1233us) server_negotiation.cc:407] Connection header received
0927 22:58:20.380277 (+ 2374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:20.380283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:20.380370 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:20.380510 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:20.383082 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:20.384107 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:20.386059 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:20.386896 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:20.387085 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:20.391437 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:20.391456 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:20.391465 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:20.391530 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:20.394959 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:20.396293 (+ 1334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:20.396297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:20.396300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:20.396458 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:20.397133 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:20.397137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:20.397139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:20.397471 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:20.397638 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:20.398848 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0927 22:58:20.399136 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":275,"threads_started":1}
I20260927 22:58:20.401646 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:20.376696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:41437)
0927 22:58:20.377669 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:20.377727 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:20.379833 (+ 2106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:20.380934 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:20.380948 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:20.381921 (+ 973us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:20.382880 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:20.382902 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:20.384316 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:20.384323 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:20.385870 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:20.385884 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:20.387173 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:20.387178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:20.387440 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:20.388345 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:20.388372 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:20.391097 (+ 2725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:20.395181 (+ 4084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:20.395197 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:20.395226 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:20.396086 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:20.396612 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:20.396620 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:20.396627 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:20.396941 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:20.397639 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:20.397652 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:20.398254 (+ 602us) client_negotiation.cc:769] Sending connection context
0927 22:58:20.398667 (+ 413us) client_negotiation.cc:240] Negotiation successful
0927 22:58:20.399491 (+ 824us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":373,"threads_started":1}
I20260927 22:58:20.404140 13051 heartbeater.cc:347] Connected to a master server at 127.4.74.190:46117
I20260927 22:58:20.404660 13051 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:20.405560 13051 heartbeater.cc:511] Master 127.4.74.190:46117 requested a full tablet report, sending...
I20260927 22:58:20.407603 12727 ts_manager.cc:194] Registered new tserver with Master: 61af181df2a34a9f9233cacf8461e1d4 (127.4.74.130:37297)
I20260927 22:58:20.408525 12727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:41437
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:58:20.448544 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46117
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:58:20.800694 12917 heartbeater.cc:503] Master 127.4.74.190:46117 was elected leader, sending a full tablet report...
W20260927 22:58:20.859898 13057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:20.860206 13057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:20.860301 13057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:20.867522 13057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:20.867671 13057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:20.867784 13057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:58:20.877691 13057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:20.879112 13057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:20.880573 13057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:20.895957 13063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:20.896076 13065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:20.896075 13062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:20.901484 13057 server_base.cc:1034] running on GCE node
I20260927 22:58:20.902252 13057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:20.903442 13057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:20.904702 13057 hybrid_clock.cc:648] HybridClock initialized: now 1790549900904637 us; error 58 us; skew 500 ppm
I20260927 22:58:20.905092 13057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:20 dist-test-slave-5pd3 krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549900, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:20.908828 13057 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:58:20.910862 13057 webserver.cc:460] Webserver started at http://127.4.74.131:40675/ using document root <none> and password file <none>
I20260927 22:58:20.911815 13057 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:20.911929 13057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:20.912321 13057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:20.915480 13057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "9853b4d582e442269f788b67df2c11e6"
format_stamp: "Formatted at 2026-09-27 22:58:20 on dist-test-slave-5pd3"
server_key: "ff9a934bae84300d76c969346a1cfd99"
server_key_iv: "3de654af225d864b6f28c3a1ba2746f6"
server_key_version: "encryptionkey@0"
I20260927 22:58:20.916548 13057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "9853b4d582e442269f788b67df2c11e6"
format_stamp: "Formatted at 2026-09-27 22:58:20 on dist-test-slave-5pd3"
server_key: "ff9a934bae84300d76c969346a1cfd99"
server_key_iv: "3de654af225d864b6f28c3a1ba2746f6"
server_key_version: "encryptionkey@0"
I20260927 22:58:20.922806 13057 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260927 22:58:20.927434 13072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:20.929652 13057 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260927 22:58:20.929912 13057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "9853b4d582e442269f788b67df2c11e6"
format_stamp: "Formatted at 2026-09-27 22:58:20 on dist-test-slave-5pd3"
server_key: "ff9a934bae84300d76c969346a1cfd99"
server_key_iv: "3de654af225d864b6f28c3a1ba2746f6"
server_key_version: "encryptionkey@0"
I20260927 22:58:20.930162 13057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:20.965049 13057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:20.969594 13057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:20.969944 13057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:20.971184 13057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:20.972790 13057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:20.972893 13057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:20.973063 13057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:20.973135 13057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:21.012149 13057 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:42433
I20260927 22:58:21.012251 13184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:42433 every 8 connection(s)
I20260927 22:58:21.013602 13057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:58:21.016260 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 13057
I20260927 22:58:21.016486 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:58:21.017383 4394 external_mini_cluster.cc:1428] Setting key d5b0b96184ae1a275ce3431e4036d7b3
Sep 27 22:58:21 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549900, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:58:21.040263 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:21.017939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:47127 (local address 127.4.74.190:46117)
0927 22:58:21.018525 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:21.018531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:21.020556 (+ 2025us) server_negotiation.cc:407] Connection header received
0927 22:58:21.022847 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:21.022851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:21.022938 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:21.023059 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:21.025004 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:21.025908 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:21.027973 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:21.028444 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:21.028569 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:21.032661 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:21.032669 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:21.032674 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:21.032730 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:21.036890 (+ 4160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:21.037714 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:21.037720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:21.037724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:21.037858 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:21.038294 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:21.038299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:21.038303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:21.038607 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:21.038789 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:21.039588 (+ 799us) server_negotiation.cc:299] Negotiation successful
0927 22:58:21.039832 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":289,"threads_started":1}
I20260927 22:58:21.041249 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:21.018839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:47127)
0927 22:58:21.020301 (+ 1462us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:21.020396 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:21.022483 (+ 2087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:21.023417 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:21.023427 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:21.024099 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:21.024759 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:21.024777 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:21.026109 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:21.026114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:21.027763 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:21.027777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:21.028704 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:21.028710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:21.028941 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:21.029771 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:21.029801 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:21.032324 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:21.037077 (+ 4753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:21.037087 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:21.037109 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:21.037572 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:21.037981 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:21.037985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:21.037994 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:21.038172 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:21.038742 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:21.038751 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:21.039151 (+ 400us) client_negotiation.cc:769] Sending connection context
0927 22:58:21.039462 (+ 311us) client_negotiation.cc:240] Negotiation successful
0927 22:58:21.040077 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"thread_start_us":737,"threads_started":1}
I20260927 22:58:21.043251 13185 heartbeater.cc:347] Connected to a master server at 127.4.74.190:46117
I20260927 22:58:21.043690 13185 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:21.044574 13185 heartbeater.cc:511] Master 127.4.74.190:46117 requested a full tablet report, sending...
I20260927 22:58:21.046497 12727 ts_manager.cc:194] Registered new tserver with Master: 9853b4d582e442269f788b67df2c11e6 (127.4.74.131:42433)
I20260927 22:58:21.047791 12727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:47127
I20260927 22:58:21.049377 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab.
Entry for principal oryx/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab.
WARNING: no policy specified for oryx/127.4.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab.
Entry for principal oryx/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab.
WARNING: no policy specified for oryx/127.4.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab.
Entry for principal oryx/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab.
I20260927 22:58:21.219043 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 12789
Sep 27 22:58:21 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:58:21.239625 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:21.220882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45100 (local address 127.4.74.129:42661)
0927 22:58:21.222082 (+ 1200us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:21.222091 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:21.222122 (+ 31us) server_negotiation.cc:407] Connection header received
0927 22:58:21.222197 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:21.222206 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:21.222497 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:21.222656 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:21.224228 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:21.225336 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:21.226738 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:21.227405 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:21.227521 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:21.232504 (+ 4983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:21.232517 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:21.232536 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:21.232582 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:21.236053 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:21.237060 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:21.237070 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:21.237073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:21.237308 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:21.237827 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:21.237832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:21.237834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:21.238289 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:21.238580 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:21.238925 (+ 345us) server_negotiation.cc:299] Negotiation successful
0927 22:58:21.239200 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1014,"thread_start_us":839,"threads_started":1}
I20260927 22:58:21.241117 12912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:21.327468 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 12789
I20260927 22:58:21.346324 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 12923
Sep 27 22:58:21 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:58:21.365465 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:21.347407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.4.74.130:37297)
0927 22:58:21.348091 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:21.348099 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:21.348129 (+ 30us) server_negotiation.cc:407] Connection header received
0927 22:58:21.348202 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:21.348209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:21.348444 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:21.348606 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:21.350152 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:21.351201 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:21.352796 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:21.353400 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:21.353523 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:21.359093 (+ 5570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:21.359108 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:21.359124 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:21.359163 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:21.362412 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:21.363276 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:21.363283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:21.363286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:21.363449 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:21.363960 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:21.363966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:21.363968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:21.364328 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:21.364611 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:21.364938 (+ 327us) server_negotiation.cc:299] Negotiation successful
0927 22:58:21.365148 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":378,"threads_started":1}
I20260927 22:58:21.366945 13046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:21.451864 13051 heartbeater.cc:503] Master 127.4.74.190:46117 was elected leader, sending a full tablet report...
I20260927 22:58:21.455777 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 12923
I20260927 22:58:21.473891 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13057
Sep 27 22:58:21 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:58:21.492493 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:21.475194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48256 (local address 127.4.74.131:42433)
0927 22:58:21.476038 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:21.476047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:21.476084 (+ 37us) server_negotiation.cc:407] Connection header received
0927 22:58:21.476168 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:21.476176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:21.476460 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:21.476632 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:21.478046 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:21.479135 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:21.480574 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:21.481265 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:21.481391 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:21.485952 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:21.485965 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:21.485984 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:21.486025 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:21.489443 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:21.490178 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:21.490187 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:21.490190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:21.490343 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:21.490745 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:21.490748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:21.490750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:21.491154 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:21.491398 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:21.491810 (+ 412us) server_negotiation.cc:299] Negotiation successful
0927 22:58:21.492097 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":418,"threads_started":1}
I20260927 22:58:21.493906 13180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:21.584918 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 13057
I20260927 22:58:21.603305 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 12695
I20260927 22:58:21.604820 12757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:21.711762 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 12695
I20260927 22:58:21.736532 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:46117
--webserver_interface=127.4.74.190
--webserver_port=43385
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:46117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:22.147399 13203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:22.147854 13203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:22.148058 13203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:22.156809 13203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:58:22.156972 13203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:22.157061 13203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:22.157160 13203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:58:22.157234 13203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:58:22.168663 13203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:46117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:46117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=43385
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:22.170529 13203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:22.172211 13203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:22.187040 13209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:22.187028 13211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:22.187907 13208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:22.188467 13203 server_base.cc:1034] running on GCE node
I20260927 22:58:22.189474 13203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:22.191325 13203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:22.192611 13203 hybrid_clock.cc:648] HybridClock initialized: now 1790549902192545 us; error 56 us; skew 500 ppm
I20260927 22:58:22.193117 13203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:22 dist-test-slave-5pd3 krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549902, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:22.202857 13203 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:58:22.205390 13203 webserver.cc:460] Webserver started at http://127.4.74.190:43385/ using document root <none> and password file <none>
I20260927 22:58:22.206394 13203 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:22.206585 13203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:22.214133 13203 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 22:58:22.218590 13218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:22.221071 13203 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260927 22:58:22.221379 13203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "9411c9826ef740ab9ac23f62f099c5b7"
format_stamp: "Formatted at 2026-09-27 22:58:18 on dist-test-slave-5pd3"
server_key: "fe0d204708ff017984e1cb5eefe12483"
server_key_iv: "135be56265eb8a9eca0d0612e86e16ae"
server_key_version: "encryptionkey@0"
I20260927 22:58:22.222807 13203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:22.248080 13203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:22.253408 13203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:22.253795 13203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:22.278303 13203 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:46117
I20260927 22:58:22.278337 13269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:46117 every 8 connection(s)
I20260927 22:58:22.280422 13203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:58:22.286618 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 13203
I20260927 22:58:22.287678 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:42661
--local_ip_for_outbound_sockets=127.4.74.129
--tserver_master_addrs=127.4.74.190:46117
--webserver_port=37793
--webserver_interface=127.4.74.129
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.74.129
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:58:22.289675 13270 sys_catalog.cc:263] Verifying existing consensus state
I20260927 22:58:22.295172 13270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7: Bootstrap starting.
I20260927 22:58:22.315034 13270 log.cc:824] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:22.324550 13270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 22:58:22.325114 13270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7: Bootstrap complete.
I20260927 22:58:22.335173 13270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } }
I20260927 22:58:22.336133 13270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9411c9826ef740ab9ac23f62f099c5b7, State: Initialized, Role: FOLLOWER
I20260927 22:58:22.336869 13270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } }
I20260927 22:58:22.337105 13270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:58:22.337230 13270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:58:22.337426 13270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 1 FOLLOWER]: Advancing to term 2
I20260927 22:58:22.341464 13270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } }
I20260927 22:58:22.342007 13270 leader_election.cc:302] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9411c9826ef740ab9ac23f62f099c5b7; no voters:
I20260927 22:58:22.342686 13270 leader_election.cc:288] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 22:58:22.342823 13274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 22:58:22.344331 13274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [term 2 LEADER]: Becoming Leader. State: Replica: 9411c9826ef740ab9ac23f62f099c5b7, State: Running, Role: LEADER
I20260927 22:58:22.344966 13274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } }
I20260927 22:58:22.345335 13270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:58:22.348344 13276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9411c9826ef740ab9ac23f62f099c5b7. Latest consensus state: current_term: 2 leader_uuid: "9411c9826ef740ab9ac23f62f099c5b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } } }
I20260927 22:58:22.348896 13276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:22.349164 13275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9411c9826ef740ab9ac23f62f099c5b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9411c9826ef740ab9ac23f62f099c5b7" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46117 } } }
I20260927 22:58:22.350411 13275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:22.352873 13282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:58:22.361370 13282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:58:22.363286 13282 catalog_manager.cc:1259] Loaded cluster ID: cfbc168b2f8c478d8568ceaa16861ea2
I20260927 22:58:22.363405 13282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:58:22.369105 13282 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:58:22.371186 13282 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9411c9826ef740ab9ac23f62f099c5b7: Loaded TSK: 0
I20260927 22:58:22.373499 13282 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 22:58:22.691090 13272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:22.691464 13272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:22.691687 13272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:22.699301 13272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:22.699487 13272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:22.699720 13272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:58:22.710036 13272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.129.keytab
--principal=oryx/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:42661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=37793
--tserver_master_addrs=127.4.74.190:46117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:22.711524 13272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:22.713047 13272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:22.728683 13297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:22.729343 13299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:22.729987 13272 server_base.cc:1034] running on GCE node
W20260927 22:58:22.731828 13296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:22.732661 13272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:22.733750 13272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:22.734977 13272 hybrid_clock.cc:648] HybridClock initialized: now 1790549902734940 us; error 36 us; skew 500 ppm
I20260927 22:58:22.735307 13272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:22 dist-test-slave-5pd3 krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549902, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:22.739019 13272 init.cc:377] Logged in from keytab as oryx/127.4.74.129@KRBTEST.COM (short username oryx)
I20260927 22:58:22.740798 13272 webserver.cc:460] Webserver started at http://127.4.74.129:37793/ using document root <none> and password file <none>
I20260927 22:58:22.741549 13272 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:22.741703 13272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:22.748160 13272 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260927 22:58:22.751849 13306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:22.753620 13272 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:58:22.753830 13272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "c85130c447744c2aab384cb0a88e327d"
format_stamp: "Formatted at 2026-09-27 22:58:19 on dist-test-slave-5pd3"
server_key: "9f63494a75f8d75febd24095d9c2cfc6"
server_key_iv: "2a7e8023789e5e72b84e4d03219a1e53"
server_key_version: "encryptionkey@0"
I20260927 22:58:22.754854 13272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:22.817073 13272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:22.822391 13272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:22.822643 13272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:22.823755 13272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:22.825194 13272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:22.825284 13272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:22.825358 13272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:22.825419 13272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:22.869076 13272 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:42661
I20260927 22:58:22.869387 13418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:42661 every 8 connection(s)
I20260927 22:58:22.870720 13272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:58:22.879185 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 13272
I20260927 22:58:22.880035 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:37297
--local_ip_for_outbound_sockets=127.4.74.130
--tserver_master_addrs=127.4.74.190:46117
--webserver_port=36445
--webserver_interface=127.4.74.130
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.74.130
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 22:58:22 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:22 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 22:58:22.892102 13421 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:22.874917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:51557 (local address 127.4.74.190:46117)
0927 22:58:22.876013 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:22.876034 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:22.876633 (+ 599us) server_negotiation.cc:407] Connection header received
0927 22:58:22.879484 (+ 2851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:22.879497 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:22.879937 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:22.880325 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:22.881964 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.883120 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.884805 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.885425 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.885629 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:22.891050 (+ 5421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:51557: BlockingRecv error: recv got EOF from 127.4.74.129:51557 (error 108)
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":496,"threads_started":1}
I20260927 22:58:22.892117 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:22.875106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:51557)
0927 22:58:22.876423 (+ 1317us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:22.876468 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:22.878816 (+ 2348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:22.880589 (+ 1773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:22.880599 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:22.881200 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:22.881749 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:22.881767 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.883297 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.883303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:22.884639 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:22.884653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.885678 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.885685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:22.885949 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:22.886828 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:22.886850 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:22.890699 (+ 3849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":736,"threads_started":1}
W20260927 22:58:22.893350 13419 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:46117 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.74.190:46117: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Sep 27 22:58:22 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:22 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:22.904985 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:22.894333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:49767)
0927 22:58:22.894554 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:22.894580 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:22.894705 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:22.895386 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:22.895391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:22.895856 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:22.896041 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:22.896051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.897952 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.897957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:22.898938 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:22.898952 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.900218 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.900223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:22.900388 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:22.901196 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:22.901228 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:22.904765 (+ 3537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260927 22:58:22.906159 13421 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:22.894499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:49767 (local address 127.4.74.190:46117)
0927 22:58:22.894761 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:22.894771 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:22.894804 (+ 33us) server_negotiation.cc:407] Connection header received
0927 22:58:22.894913 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:22.894922 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:22.895039 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:22.895213 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:22.896252 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.897722 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.899119 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.900000 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.900183 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:22.905684 (+ 5501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:49767: BlockingRecv error: recv got EOF from 127.4.74.129:49767 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Sep 27 22:58:22 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:22 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:22.916057 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:22.905972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:43089)
0927 22:58:22.906270 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:22.906292 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:22.906412 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:22.907085 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:22.907089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:22.907368 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:22.907521 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:22.907531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.909337 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.909341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:22.910296 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:22.910308 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.911458 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.911465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:22.911726 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:22.912496 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:22.912513 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:22.915809 (+ 3296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183}
W20260927 22:58:22.916496 13419 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 22:58:22.916999 13421 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:22.906140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:43089 (local address 127.4.74.190:46117)
0927 22:58:22.906612 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:22.906621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:22.906649 (+ 28us) server_negotiation.cc:407] Connection header received
0927 22:58:22.906760 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:22.906768 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:22.906879 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:22.907028 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:22.907771 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.909154 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.910462 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:22.911181 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:22.911340 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:22.916239 (+ 4899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:43089: BlockingRecv error: recv got EOF from 127.4.74.129:43089 (error 108)
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":496,"threads_started":1}
W20260927 22:58:23.293411 13423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:23.293771 13423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:23.293877 13423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:23.301841 13423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:23.301987 13423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:23.302120 13423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:58:23.313285 13423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.130.keytab
--principal=oryx/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:37297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=36445
--tserver_master_addrs=127.4.74.190:46117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:23.314996 13423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:23.316634 13423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:23.332996 13429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:23.332881 13432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:23.333551 13430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:23.334645 13423 server_base.cc:1034] running on GCE node
I20260927 22:58:23.335382 13423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:23.336575 13423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:23.337872 13423 hybrid_clock.cc:648] HybridClock initialized: now 1790549903337817 us; error 71 us; skew 500 ppm
I20260927 22:58:23.338277 13423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:23 dist-test-slave-5pd3 krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549903, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:23.341933 13423 init.cc:377] Logged in from keytab as oryx/127.4.74.130@KRBTEST.COM (short username oryx)
I20260927 22:58:23.343866 13423 webserver.cc:460] Webserver started at http://127.4.74.130:36445/ using document root <none> and password file <none>
I20260927 22:58:23.344669 13423 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:23.344842 13423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:23.352077 13423 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 22:58:23.355628 13439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:23.357462 13423 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:58:23.357712 13423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "61af181df2a34a9f9233cacf8461e1d4"
format_stamp: "Formatted at 2026-09-27 22:58:20 on dist-test-slave-5pd3"
server_key: "4dc7caa27c6e1e1a5f5c5ec82cf5898e"
server_key_iv: "f6b73b32819a6c52bd286d0c18be7a1e"
server_key_version: "encryptionkey@0"
I20260927 22:58:23.358790 13423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:23.389235 13423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:23.394126 13423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:23.394604 13423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:23.395953 13423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:23.397500 13423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:23.397642 13423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:23.397789 13423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:23.397881 13423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:23.436673 13423 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:37297
I20260927 22:58:23.436786 13551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:37297 every 8 connection(s)
I20260927 22:58:23.438166 13423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:58:23.444738 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 13423
I20260927 22:58:23.445518 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:42433
--local_ip_for_outbound_sockets=127.4.74.131
--tserver_master_addrs=127.4.74.190:46117
--webserver_port=40675
--webserver_interface=127.4.74.131
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.74.131
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 22:58:23 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:23 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 22:58:23.455819 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:23.441423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:35297 (local address 127.4.74.190:46117)
0927 22:58:23.441994 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:23.441999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:23.443132 (+ 1133us) server_negotiation.cc:407] Connection header received
0927 22:58:23.445239 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:23.445244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:23.445338 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:23.445475 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:23.447162 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.447990 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.449515 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.449936 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.450070 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:23.455464 (+ 5394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:35297: BlockingRecv error: recv got EOF from 127.4.74.130:35297 (error 108)
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":287,"threads_started":1}
I20260927 22:58:23.456496 13555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:23.442103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:35297)
0927 22:58:23.442925 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:23.442971 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:23.444925 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:23.445753 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:23.445766 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:23.446323 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:23.446983 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:23.447000 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.448149 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.448153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:23.449354 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:23.449365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.450173 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.450180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:23.450413 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:23.451212 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:23.451233 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:23.455257 (+ 4024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":318,"threads_started":1}
W20260927 22:58:23.457523 13552 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:46117 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.74.190:46117: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Sep 27 22:58:23 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:23 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:23.467903 13555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:23.458339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:43867)
0927 22:58:23.458604 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:23.458627 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:23.458817 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:23.459254 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:23.459259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:23.459695 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:23.459859 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:23.459867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.461032 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.461040 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:23.462183 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:23.462195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.463115 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.463122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:23.463359 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:23.464217 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:23.464244 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:23.467601 (+ 3357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107}
W20260927 22:58:23.468050 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:23.458477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:43867 (local address 127.4.74.190:46117)
0927 22:58:23.458658 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:23.458666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:23.458689 (+ 23us) server_negotiation.cc:407] Connection header received
0927 22:58:23.458985 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:23.458998 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:23.459061 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:23.459177 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:23.460047 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.460871 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.462331 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.462954 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.463124 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:23.467817 (+ 4693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:43867: BlockingRecv error: recv got EOF from 127.4.74.130:43867 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 27 22:58:23 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:23 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:23.478356 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:23.468667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:54387)
0927 22:58:23.469309 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:23.469324 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:23.469441 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:23.469905 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:23.469908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:23.470270 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:23.470473 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:23.470481 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.472232 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.472236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:23.473173 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:23.473184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.474239 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.474244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:23.474376 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:23.475065 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:23.475083 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:23.478117 (+ 3034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":322,"threads_started":1}
W20260927 22:58:23.478701 13552 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 22:58:23.478729 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:23.468956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:54387 (local address 127.4.74.190:46117)
0927 22:58:23.469221 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:23.469230 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:23.469426 (+ 196us) server_negotiation.cc:407] Connection header received
0927 22:58:23.469584 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:23.469593 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:23.469713 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:23.469906 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:23.470666 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.472077 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.473322 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.474024 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.474172 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:23.478414 (+ 4242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:54387: BlockingRecv error: recv got EOF from 127.4.74.130:54387 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260927 22:58:23.851351 13556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:23.851770 13556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:23.851955 13556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:23.859863 13556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:23.859995 13556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:23.860177 13556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:58:23.872357 13556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/krb5kdc/oryx_127.4.74.131.keytab
--principal=oryx/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:42433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=40675
--tserver_master_addrs=127.4.74.190:46117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:23.874132 13556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:23.875948 13556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:23.892253 13563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:23.892143 13562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:23.892197 13565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:23.893628 13556 server_base.cc:1034] running on GCE node
I20260927 22:58:23.894429 13556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:23.895800 13556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:23.897066 13556 hybrid_clock.cc:648] HybridClock initialized: now 1790549903897007 us; error 49 us; skew 500 ppm
I20260927 22:58:23.897511 13556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:23 dist-test-slave-5pd3 krb5kdc[12682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549903, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:23.901692 13556 init.cc:377] Logged in from keytab as oryx/127.4.74.131@KRBTEST.COM (short username oryx)
I20260927 22:58:23.903753 13556 webserver.cc:460] Webserver started at http://127.4.74.131:40675/ using document root <none> and password file <none>
I20260927 22:58:23.904692 13556 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:23.904881 13556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:23.913828 13556 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 22:58:23.918295 13572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:23.920677 13556 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260927 22:58:23.920953 13556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "9853b4d582e442269f788b67df2c11e6"
format_stamp: "Formatted at 2026-09-27 22:58:20 on dist-test-slave-5pd3"
server_key: "ff9a934bae84300d76c969346a1cfd99"
server_key_iv: "3de654af225d864b6f28c3a1ba2746f6"
server_key_version: "encryptionkey@0"
I20260927 22:58:23.922282 13556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 27 22:58:23 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:23 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:23.929253 13573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:23.917505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:51727)
0927 22:58:23.919190 (+ 1685us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:23.919214 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:23.919420 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:23.920052 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:23.920056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:23.920529 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:23.920773 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:23.920814 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.921960 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.921966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:23.923270 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:23.923288 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.924820 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.924826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:23.925035 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:23.925810 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:23.925829 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:23.928996 (+ 3167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1527,"thread_start_us":1386,"threads_started":1}
W20260927 22:58:23.929946 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:23.917624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:51727 (local address 127.4.74.190:46117)
0927 22:58:23.917884 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:23.917891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:23.919387 (+ 1496us) server_negotiation.cc:407] Connection header received
0927 22:58:23.919644 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:23.919651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:23.919796 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:23.919955 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:23.921051 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.921797 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.923524 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:23.924422 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:23.924661 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:23.929377 (+ 4716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:51727: BlockingRecv error: recv got EOF from 127.4.74.129:51727 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260927 22:58:23.947182 13556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:23.952544 13556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:23.952828 13556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:23.954042 13556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:23.955662 13556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:23.955812 13556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:23.955964 13556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:23.956055 13556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:23.998638 13556 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:42433
I20260927 22:58:23.998710 13685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:42433 every 8 connection(s)
I20260927 22:58:24.000563 13556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:58:24.007123 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 13556
Sep 27 22:58:24 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:24 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 22:58:24.023005 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:24.003953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:37315 (local address 127.4.74.190:46117)
0927 22:58:24.004172 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:24.004177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:24.005404 (+ 1227us) server_negotiation.cc:407] Connection header received
0927 22:58:24.007937 (+ 2533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:24.007947 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:24.008080 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:24.008211 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:24.011089 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.012849 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.014944 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.015663 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.015811 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.022649 (+ 6838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:37315: BlockingRecv error: recv got EOF from 127.4.74.131:37315 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 22:58:24.024106 13688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:24.004307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:37315)
0927 22:58:24.005181 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:24.005229 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:24.007423 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:24.008634 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:24.008650 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:24.009512 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:24.010330 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:24.010351 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.013096 (+ 2745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.013101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:24.014745 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:24.014763 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.015908 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.015914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:24.016279 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.017364 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:24.017427 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:24.022387 (+ 4960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":344,"threads_started":1}
W20260927 22:58:24.025004 13686 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:46117 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.74.190:46117: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
I20260927 22:58:24.027879 13689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:24.008269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38540 (local address 127.4.74.190:46117)
0927 22:58:24.009018 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:24.009024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:24.009372 (+ 348us) server_negotiation.cc:407] Connection header received
0927 22:58:24.009626 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:24.009631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:24.009702 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:24.009926 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:24.013276 (+ 3350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.014132 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.016035 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.016702 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.016851 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.019966 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:24.019975 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:24.019993 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:24.020035 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:24.023759 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:24.024572 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:24.024581 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:24.024589 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:24.024784 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:24.025311 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:24.025320 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:24.025322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:24.025982 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:24.026312 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:24.026996 (+ 684us) server_negotiation.cc:299] Negotiation successful
0927 22:58:24.027409 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":328,"threads_started":1}
Sep 27 22:58:24 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:24 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:24.035838 13688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:24.025688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:38177)
0927 22:58:24.025973 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:24.025997 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:24.026129 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:24.026558 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:24.026562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:24.027058 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:24.027250 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:24.027261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.028772 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.028778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:24.029947 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:24.029960 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.031043 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.031049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:24.031756 (+ 707us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.032579 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:24.032610 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:24.035620 (+ 3010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101}
W20260927 22:58:24.036339 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:24.025861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:38177 (local address 127.4.74.190:46117)
0927 22:58:24.026150 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:24.026159 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:24.026184 (+ 25us) server_negotiation.cc:407] Connection header received
0927 22:58:24.026254 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:24.026259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:24.026390 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:24.026516 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:24.027504 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.028618 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.030092 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.030855 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.031024 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.036041 (+ 5017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:38177: BlockingRecv error: recv got EOF from 127.4.74.131:38177 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
Sep 27 22:58:24 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:24 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:24.046001 13688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:24.036698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:43161)
0927 22:58:24.036884 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:24.036910 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:24.037006 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:24.037599 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:24.037603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:24.037904 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:24.038086 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:24.038096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.039397 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.039402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:24.040608 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:24.040623 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.041491 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.041495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:24.041653 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.042367 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:24.042387 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:24.045664 (+ 3277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260927 22:58:24.046499 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:24.036804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:43161 (local address 127.4.74.190:46117)
0927 22:58:24.036966 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:24.036971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:24.036989 (+ 18us) server_negotiation.cc:407] Connection header received
0927 22:58:24.037126 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:24.037130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:24.037208 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:24.037315 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:24.038314 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.039182 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.040796 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.041290 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.041416 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.045998 (+ 4582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:43161: BlockingRecv error: recv got EOF from 127.4.74.131:43161 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260927 22:58:24.046521 13686 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 22:58:24 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:24 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:24.488909 13691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:24.479432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:33243)
0927 22:58:24.480062 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:24.480079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:24.480212 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:24.480651 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:24.480654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:24.481093 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:24.481272 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:24.481281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.482365 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.482378 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:24.483368 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:24.483380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.484285 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.484290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:24.484460 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.485187 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:24.485204 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:24.488683 (+ 3479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":377,"threads_started":1}
W20260927 22:58:24.489287 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:24.479664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:33243 (local address 127.4.74.190:46117)
0927 22:58:24.479886 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:24.479896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:24.480173 (+ 277us) server_negotiation.cc:407] Connection header received
0927 22:58:24.480393 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:24.480398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:24.480478 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:24.480597 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:24.481423 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.482213 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.483578 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.484091 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.484221 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.488998 (+ 4777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:33243: BlockingRecv error: recv got EOF from 127.4.74.130:33243 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 27 22:58:24 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:24 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:24.943866 13692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:24.931448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:46975)
0927 22:58:24.932193 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:24.932211 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:24.932363 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:24.932885 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:24.932889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:24.933329 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:24.933490 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:24.933500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.934790 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.934794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:24.936029 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:24.936045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.937775 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.937782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:24.937964 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.938669 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:24.938687 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:24.943350 (+ 4663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":484,"threads_started":1}
W20260927 22:58:24.944027 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:24.931746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:46975 (local address 127.4.74.190:46117)
0927 22:58:24.931951 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:24.931955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:24.932381 (+ 426us) server_negotiation.cc:407] Connection header received
0927 22:58:24.932483 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:24.932500 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:24.932615 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:24.932749 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:24.933783 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.934616 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.936496 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:24.937444 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:24.937786 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:24.943742 (+ 5956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:46975: BlockingRecv error: recv got EOF from 127.4.74.129:46975 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 27 22:58:25 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:25 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:25.058985 13693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:25.047338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:33761)
0927 22:58:25.047948 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:25.047966 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:25.048106 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:25.048769 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:25.048773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:25.049316 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:25.049559 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:25.049569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.050769 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.050774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:25.052033 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:25.052049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.053072 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.053077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:25.053218 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:25.053964 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:25.053984 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:25.058538 (+ 4554us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":346,"threads_started":1}
W20260927 22:58:25.059095 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:25.047471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:33761 (local address 127.4.74.190:46117)
0927 22:58:25.047770 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:25.047775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:25.048085 (+ 310us) server_negotiation.cc:407] Connection header received
0927 22:58:25.048468 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:25.048472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:25.048554 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:25.048674 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:25.049762 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.050590 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.052252 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.052775 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.052917 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:25.058832 (+ 5915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:33761: BlockingRecv error: recv got EOF from 127.4.74.131:33761 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 27 22:58:25 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:25 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:25.501030 13694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:25.490021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:56731)
0927 22:58:25.490727 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:25.490746 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:25.490984 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:25.491371 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:25.491376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:25.492055 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:25.492239 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:25.492249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.493417 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.493425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:25.494557 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:25.494573 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.495862 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.495868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:25.496057 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:25.496865 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:25.496889 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:25.500739 (+ 3850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":404,"threads_started":1}
W20260927 22:58:25.501396 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:25.490196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:56731 (local address 127.4.74.190:46117)
0927 22:58:25.490527 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:25.490535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:25.490883 (+ 348us) server_negotiation.cc:407] Connection header received
0927 22:58:25.491134 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:25.491140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:25.491221 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:25.491340 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:25.492424 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.493244 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.494701 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.495683 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.495826 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:25.501062 (+ 5236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:56731: BlockingRecv error: recv got EOF from 127.4.74.130:56731 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Sep 27 22:58:25 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:25 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:25.955799 13695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:25.945237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:34963)
0927 22:58:25.945961 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:25.945980 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:25.946128 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:25.946642 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:25.946645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:25.947082 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:25.947359 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:25.947369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.948738 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.948744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:25.949742 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:25.949757 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.950557 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.950563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:25.950759 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:25.951460 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:25.951478 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:25.955251 (+ 3773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":382,"threads_started":1}
W20260927 22:58:25.955947 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:25.945445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:34963 (local address 127.4.74.190:46117)
0927 22:58:25.945666 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:25.945670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:25.946072 (+ 402us) server_negotiation.cc:407] Connection header received
0927 22:58:25.946331 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:25.946336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:25.946435 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:25.946573 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:25.947637 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.948581 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.949903 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:25.950386 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:25.950556 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:25.955604 (+ 5048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:34963: BlockingRecv error: recv got EOF from 127.4.74.129:34963 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Sep 27 22:58:26 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:26 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:26.070204 13696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:26.060492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:47101)
0927 22:58:26.061097 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:26.061116 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:26.061316 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:26.061700 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:26.061709 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:26.062192 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:26.062363 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:26.062373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.063619 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.063629 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:26.064842 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:26.064855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.065601 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.065608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:26.065792 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:26.066482 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:26.066503 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:26.069789 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":338,"threads_started":1}
W20260927 22:58:26.070353 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:26.060600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:47101 (local address 127.4.74.190:46117)
0927 22:58:26.060879 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:26.060889 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:26.061215 (+ 326us) server_negotiation.cc:407] Connection header received
0927 22:58:26.061455 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:26.061461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:26.061545 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:26.061658 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:26.062514 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.063394 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.064998 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.065418 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.065560 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:26.070063 (+ 4503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:47101: BlockingRecv error: recv got EOF from 127.4.74.131:47101 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Sep 27 22:58:26 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:26 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:26.512544 13697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:26.502470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:34443)
0927 22:58:26.503096 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:26.503114 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:26.503297 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:26.503863 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:26.503870 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:26.504472 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:26.504722 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:26.504734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.505925 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.505930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:26.506973 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:26.506987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.507988 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.507994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:26.508160 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:26.508912 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:26.508935 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:26.512226 (+ 3291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":354,"threads_started":1}
W20260927 22:58:26.512722 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:26.502599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:34443 (local address 127.4.74.190:46117)
0927 22:58:26.502854 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:26.502861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:26.503221 (+ 360us) server_negotiation.cc:407] Connection header received
0927 22:58:26.503444 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:26.503470 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:26.503599 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:26.503725 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:26.504915 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.505690 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.507158 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.507721 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.507845 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:26.512474 (+ 4629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:34443: BlockingRecv error: recv got EOF from 127.4.74.130:34443 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 27 22:58:26 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:26 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:26.967453 13698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:26.957271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:51887)
0927 22:58:26.957909 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:26.957930 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:26.958098 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:26.958563 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:26.958566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:26.959047 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:26.959222 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:26.959230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.960567 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.960572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:26.961672 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:26.961686 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.962643 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.962648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:26.962811 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:26.963665 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:26.963684 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:26.967134 (+ 3450us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":344,"threads_started":1}
W20260927 22:58:26.967799 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:26.957343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:51887 (local address 127.4.74.190:46117)
0927 22:58:26.957649 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:26.957658 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:26.958076 (+ 418us) server_negotiation.cc:407] Connection header received
0927 22:58:26.958262 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:26.958268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:26.958391 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:26.958486 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:26.959390 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.960368 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.961849 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:26.962409 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:26.962546 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:26.967441 (+ 4895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:51887: BlockingRecv error: recv got EOF from 127.4.74.129:51887 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Sep 27 22:58:27 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:27 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:27.081605 13699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:27.071485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:51115)
0927 22:58:27.072199 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:27.072218 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:27.072431 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:27.072810 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:27.072815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:27.073248 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:27.073464 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:27.073473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.074602 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.074610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:27.075806 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:27.075819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.076577 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.076584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:27.076787 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:27.077553 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:27.077571 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:27.081170 (+ 3599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":376,"threads_started":1}
W20260927 22:58:27.081820 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:27.071709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:51115 (local address 127.4.74.190:46117)
0927 22:58:27.072054 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:27.072061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:27.072334 (+ 273us) server_negotiation.cc:407] Connection header received
0927 22:58:27.072590 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:27.072595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:27.072666 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:27.072777 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:27.073639 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.074435 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.075965 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.076407 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.076555 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:27.081430 (+ 4875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:51115: BlockingRecv error: recv got EOF from 127.4.74.131:51115 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
Sep 27 22:58:27 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:27 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:27.531736 13700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:27.513848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:38519)
0927 22:58:27.514639 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:27.514677 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:27.514905 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:27.515304 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:27.515309 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:27.516074 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:27.516288 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:27.516298 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.517476 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.517483 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:27.518584 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:27.518596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.519363 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.519369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:27.519628 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:27.526864 (+ 7236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:27.526890 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:27.531291 (+ 4401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":339,"threads_started":1}
W20260927 22:58:27.531899 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:27.514115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:38519 (local address 127.4.74.190:46117)
0927 22:58:27.514400 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:27.514408 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:27.514851 (+ 443us) server_negotiation.cc:407] Connection header received
0927 22:58:27.515060 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:27.515068 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:27.515144 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:27.515260 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:27.516441 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.517315 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.518734 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.519199 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.519335 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:27.531605 (+ 12270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:38519: BlockingRecv error: recv got EOF from 127.4.74.130:38519 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Sep 27 22:58:27 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:27 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:27.978937 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:27.968668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:48573)
0927 22:58:27.969320 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:27.969340 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:27.969497 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:27.969924 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:27.969928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:27.970415 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:27.970633 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:27.970644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.971789 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.971794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:27.972886 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:27.972903 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.973710 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.973717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:27.973896 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:27.974694 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:27.974715 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:27.978541 (+ 3826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":341,"threads_started":1}
W20260927 22:58:27.979041 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:27.968788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:48573 (local address 127.4.74.190:46117)
0927 22:58:27.969034 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:27.969039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:27.969462 (+ 423us) server_negotiation.cc:407] Connection header received
0927 22:58:27.969630 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:27.969634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:27.969725 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:27.969832 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:27.970790 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.971609 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.973073 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:27.973488 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:27.973619 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:27.978771 (+ 5152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:48573: BlockingRecv error: recv got EOF from 127.4.74.129:48573 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Sep 27 22:58:28 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:28 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:28.093063 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:28.082738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:56699)
0927 22:58:28.083357 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:28.083376 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:28.083516 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:28.083959 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:28.083963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:28.084398 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:28.084603 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:28.084613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.085763 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.085768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:28.086868 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:28.086882 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.087846 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.087851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:28.088006 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:28.088771 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:28.088792 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:28.092684 (+ 3892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":351,"threads_started":1}
W20260927 22:58:28.093321 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:28.082808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:56699 (local address 127.4.74.190:46117)
0927 22:58:28.082998 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:28.083003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:28.083503 (+ 500us) server_negotiation.cc:407] Connection header received
0927 22:58:28.083666 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:28.083671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:28.083742 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:28.083848 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:28.084759 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.085606 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.087028 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.087518 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.087745 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:28.092967 (+ 5222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:56699: BlockingRecv error: recv got EOF from 127.4.74.131:56699 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 27 22:58:28 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:28 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:28.543326 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:28.533188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:35883)
0927 22:58:28.533943 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:28.533965 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:28.534192 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:28.534659 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:28.534662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:28.535112 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:28.535314 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:28.535324 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.536458 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.536466 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:28.537804 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:28.537821 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.538847 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.538853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:28.539003 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:28.539781 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:28.539801 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:28.543046 (+ 3245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":404,"threads_started":1}
W20260927 22:58:28.543632 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:28.533525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:35883 (local address 127.4.74.190:46117)
0927 22:58:28.533772 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:28.533781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:28.534122 (+ 341us) server_negotiation.cc:407] Connection header received
0927 22:58:28.534383 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:28.534388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:28.534457 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:28.534580 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:28.535486 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.536278 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.537997 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.538677 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.538817 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:28.543314 (+ 4497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:35883: BlockingRecv error: recv got EOF from 127.4.74.130:35883 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Sep 27 22:58:28 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:28 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:28.989775 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:28.980277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:44575)
0927 22:58:28.980978 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:28.980997 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:28.981198 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:28.981616 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:28.981621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:28.982117 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:28.982335 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:28.982350 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.983727 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.983738 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:28.984811 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:28.984823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.985569 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.985576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:28.985736 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:28.986413 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:28.986431 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:28.989543 (+ 3112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":427,"threads_started":1}
W20260927 22:58:28.990365 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:28.980495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:44575 (local address 127.4.74.190:46117)
0927 22:58:28.980784 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:28.980792 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:28.981108 (+ 316us) server_negotiation.cc:407] Connection header received
0927 22:58:28.981343 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:28.981350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:28.981433 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:28.981543 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:28.982492 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.983587 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.984957 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:28.985404 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:28.985540 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:28.990066 (+ 4526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:44575: BlockingRecv error: recv got EOF from 127.4.74.129:44575 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Sep 27 22:58:29 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:29 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:29.104470 13705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:29.094377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:39715)
0927 22:58:29.095188 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:29.095233 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:29.095444 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:29.095844 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:29.095849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:29.096310 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:29.096515 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:29.096525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.097681 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.097689 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:29.099023 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:29.099065 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.099939 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.099946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:29.100133 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:29.100787 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:29.100804 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:29.104111 (+ 3307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":433,"threads_started":1}
W20260927 22:58:29.104592 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:29.094632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:39715 (local address 127.4.74.190:46117)
0927 22:58:29.094948 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:29.094956 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:29.095357 (+ 401us) server_negotiation.cc:407] Connection header received
0927 22:58:29.095629 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:29.095634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:29.095696 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:29.095811 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:29.096695 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.097536 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.099200 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.099747 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.099900 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:29.104345 (+ 4445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:39715: BlockingRecv error: recv got EOF from 127.4.74.131:39715 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
Sep 27 22:58:29 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:29 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:29.553287 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:29.544429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:46385)
0927 22:58:29.545050 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:29.545069 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:29.545302 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:29.545653 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:29.545658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:29.546089 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:29.546236 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:29.546244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.547275 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.547281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:29.548215 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:29.548226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.548949 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.548955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:29.549115 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:29.549861 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:29.549907 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:29.552952 (+ 3045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":368,"threads_started":1}
W20260927 22:58:29.553472 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:29.544498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:46385 (local address 127.4.74.190:46117)
0927 22:58:29.544786 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:29.544795 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:29.545190 (+ 395us) server_negotiation.cc:407] Connection header received
0927 22:58:29.545450 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:29.545455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:29.545515 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:29.545621 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:29.546409 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.547120 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.548357 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.548789 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.548924 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:29.553184 (+ 4260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:46385: BlockingRecv error: recv got EOF from 127.4.74.130:46385 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 27 22:58:29 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:29 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:30.001324 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:29.990990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:35645)
0927 22:58:29.991787 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:29.991807 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:29.991969 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:29.992467 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:29.992471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:29.992947 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:29.993156 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:29.993166 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.994378 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.994383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:29.995461 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:29.995475 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.996314 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.996318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:29.996447 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:29.997151 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:29.997170 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:30.001022 (+ 3852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":357,"threads_started":1}
W20260927 22:58:30.001526 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:29.991073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:35645 (local address 127.4.74.190:46117)
0927 22:58:29.991350 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:29.991355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:29.992054 (+ 699us) server_negotiation.cc:407] Connection header received
0927 22:58:29.992159 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:29.992164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:29.992239 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:29.992363 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:29.993352 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.994176 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.995649 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:29.996121 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:29.996245 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:30.001288 (+ 5043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:35645: BlockingRecv error: recv got EOF from 127.4.74.129:35645 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Sep 27 22:58:30 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:30 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:30.117365 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:30.105765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:51031)
0927 22:58:30.106506 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:30.106534 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:30.106761 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:30.107275 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:30.107282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:30.107888 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:30.108152 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:30.108166 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:30.109722 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:30.109731 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:30.111143 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:30.111160 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:30.112687 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:30.112694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:30.112853 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:30.113650 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:30.113671 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:30.117084 (+ 3413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":344,"threads_started":1}
W20260927 22:58:30.117710 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:30.105997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:51031 (local address 127.4.74.190:46117)
0927 22:58:30.106244 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:30.106252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:30.106679 (+ 427us) server_negotiation.cc:407] Connection header received
0927 22:58:30.106925 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:30.106931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:30.107049 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:30.107184 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:30.108359 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:30.109471 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:30.111357 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:30.112106 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:30.112330 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:30.117369 (+ 5039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:51031: BlockingRecv error: recv got EOF from 127.4.74.131:51031 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Sep 27 22:58:30 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:30 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:30.563920 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:30.554802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:54923)
0927 22:58:30.555659 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:30.555714 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:30.555881 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:30.556255 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:30.556260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:30.556703 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:30.556863 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:30.556872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:30.557875 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:30.557881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:30.558897 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:30.558915 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:30.559613 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:30.559620 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:30.559789 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:30.560468 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:30.560484 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:30.563430 (+ 2946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":322,"threads_started":1}
W20260927 22:58:30.564049 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:30.554912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:54923 (local address 127.4.74.190:46117)
0927 22:58:30.555252 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:30.555261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:30.555804 (+ 543us) server_negotiation.cc:407] Connection header received
0927 22:58:30.556023 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:30.556027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:30.556117 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:30.556232 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:30.557009 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:30.557739 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:30.559034 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:30.559424 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:30.559558 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:30.563762 (+ 4204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:54923: BlockingRecv error: recv got EOF from 127.4.74.130:54923 (error 108)
Metrics: {"server-negotiator.queue_time_us":152}
Sep 27 22:58:31 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:31 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:31.013243 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:31.002883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:47587)
0927 22:58:31.003537 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:31.003590 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:31.003739 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:31.004164 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:31.004169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:31.004605 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:31.004804 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:31.004814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.006297 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.006302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:31.007664 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:31.007680 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.008540 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.008546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:31.008685 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:31.009388 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:31.009408 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:31.012959 (+ 3551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":343,"threads_started":1}
W20260927 22:58:31.013630 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:31.003044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:47587 (local address 127.4.74.190:46117)
0927 22:58:31.003252 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:31.003257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:31.003694 (+ 437us) server_negotiation.cc:407] Connection header received
0927 22:58:31.003876 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:31.003879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:31.003944 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:31.004080 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:31.005044 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.006139 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.007813 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.008274 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.008381 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:31.013296 (+ 4915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:47587: BlockingRecv error: recv got EOF from 127.4.74.129:47587 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Sep 27 22:58:31 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:31 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 22:58:31.129806 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:31.118993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:46341 (local address 127.4.74.190:46117)
0927 22:58:31.119228 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:31.119234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:31.119702 (+ 468us) server_negotiation.cc:407] Connection header received
0927 22:58:31.119924 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:31.119928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:31.120001 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:31.120088 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:31.121257 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.122072 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.123413 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.123871 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.123990 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:31.129575 (+ 5585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:46341: BlockingRecv error: recv got EOF from 127.4.74.131:46341 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 22:58:31.129827 13711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:31.118584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:46341)
0927 22:58:31.119485 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:31.119527 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:31.119745 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:31.120181 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:31.120185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:31.120881 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:31.121113 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:31.121122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.122217 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.122221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:31.123239 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:31.123251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.124034 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.124039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:31.124169 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:31.124881 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:31.124900 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:31.129203 (+ 4303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":333,"threads_started":1}
Sep 27 22:58:31 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:31 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:31.574821 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:31.564997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:58485)
0927 22:58:31.565620 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:31.565638 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:31.565852 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:31.566229 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:31.566234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:31.566746 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:31.566968 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:31.566978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.568402 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.568409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:31.569448 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:31.569462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.570279 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.570286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:31.570478 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:31.571187 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:31.571205 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:31.574446 (+ 3241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":291,"threads_started":1}
W20260927 22:58:31.575018 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:31.565116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:58485 (local address 127.4.74.190:46117)
0927 22:58:31.565376 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:31.565382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:31.565756 (+ 374us) server_negotiation.cc:407] Connection header received
0927 22:58:31.565993 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:31.565999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:31.566079 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:31.566195 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:31.567131 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.568236 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.569622 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:31.570076 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:31.570223 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:31.574690 (+ 4467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:58485: BlockingRecv error: recv got EOF from 127.4.74.130:58485 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 27 22:58:32 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:32 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:32.025856 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:32.014725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:39107)
0927 22:58:32.015339 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:32.015357 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:32.015597 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:32.016103 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:32.016108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:32.016791 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:32.017009 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:32.017022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.018354 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.018361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:32.019790 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:32.019812 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.020741 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.020747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:32.020966 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:32.021718 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:32.021740 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:32.025348 (+ 3608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":329,"threads_started":1}
W20260927 22:58:32.025918 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:32.014985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:39107 (local address 127.4.74.190:46117)
0927 22:58:32.015274 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:32.015279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:32.015534 (+ 255us) server_negotiation.cc:407] Connection header received
0927 22:58:32.015776 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:32.015781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:32.015866 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:32.016067 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:32.017190 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.018049 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.020020 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.020533 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.020708 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:32.025646 (+ 4938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:39107: BlockingRecv error: recv got EOF from 127.4.74.129:39107 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Sep 27 22:58:32 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:32 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:32.140645 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:32.131111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:47641)
0927 22:58:32.131750 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:32.131767 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:32.131971 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:32.132644 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:32.132649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:32.133213 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:32.133430 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:32.133440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.134683 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.134687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:32.135714 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:32.135727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.136492 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.136497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:32.136641 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:32.137472 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:32.137492 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:32.140371 (+ 2879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":304,"threads_started":1}
W20260927 22:58:32.141004 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:32.131309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:47641 (local address 127.4.74.190:46117)
0927 22:58:32.131614 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:32.131621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:32.131940 (+ 319us) server_negotiation.cc:407] Connection header received
0927 22:58:32.132309 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:32.132349 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:32.132447 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:32.132570 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:32.133625 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.134485 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.135848 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.136316 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.136422 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:32.140631 (+ 4209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:47641: BlockingRecv error: recv got EOF from 127.4.74.131:47641 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Sep 27 22:58:32 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:32 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:32.591950 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:32.576092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:50217)
0927 22:58:32.576727 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:32.576744 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:32.576865 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:32.577243 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:32.577246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:32.577646 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:32.577831 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:32.577840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.578921 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.578925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:32.580155 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:32.580167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.580994 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.580998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:32.581123 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:32.587876 (+ 6753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:32.587902 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:32.591600 (+ 3698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":354,"threads_started":1}
W20260927 22:58:32.592167 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:32.576214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:50217 (local address 127.4.74.190:46117)
0927 22:58:32.576409 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:32.576414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:32.576829 (+ 415us) server_negotiation.cc:407] Connection header received
0927 22:58:32.576995 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:32.576999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:32.577075 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:32.577199 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:32.577986 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.578768 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.580334 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:32.580814 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:32.580953 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:32.591901 (+ 10948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:50217: BlockingRecv error: recv got EOF from 127.4.74.130:50217 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 27 22:58:33 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:33 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:33.038175 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:33.027426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:46503)
0927 22:58:33.028186 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:33.028206 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:33.028367 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:33.028992 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:33.028998 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:33.029707 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:33.029892 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:33.029903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.031430 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.031438 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:33.032490 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:33.032505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.033417 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.033422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:33.033558 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:33.034280 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:33.034298 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:33.037750 (+ 3452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":461,"threads_started":1}
W20260927 22:58:33.038339 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:33.027536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:46503 (local address 127.4.74.190:46117)
0927 22:58:33.027863 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:33.027877 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:33.028346 (+ 469us) server_negotiation.cc:407] Connection header received
0927 22:58:33.028543 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:33.028551 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:33.028681 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:33.028893 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:33.030066 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.031235 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.032671 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.033194 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.033318 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:33.038030 (+ 4712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:46503: BlockingRecv error: recv got EOF from 127.4.74.129:46503 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 27 22:58:33 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:33 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:33.152330 13717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:33.141935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:54453)
0927 22:58:33.142543 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:33.142563 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:33.142747 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:33.143212 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:33.143215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:33.143757 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:33.144044 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:33.144055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.145299 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.145304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:33.146701 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:33.146714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.147610 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.147616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:33.147798 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:33.148649 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:33.148668 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:33.152042 (+ 3374us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":357,"threads_started":1}
W20260927 22:58:33.152631 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:33.142061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:54453 (local address 127.4.74.190:46117)
0927 22:58:33.142397 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:33.142405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:33.142740 (+ 335us) server_negotiation.cc:407] Connection header received
0927 22:58:33.142947 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:33.142952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:33.143038 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:33.143156 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:33.144201 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.145141 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.146861 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.147418 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.147595 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:33.152322 (+ 4727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:54453: BlockingRecv error: recv got EOF from 127.4.74.131:54453 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Sep 27 22:58:33 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:33 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:33.602706 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:33.593203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:50755)
0927 22:58:33.593776 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:33.593794 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:33.593927 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:33.594383 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:33.594387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:33.594800 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:33.594959 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:33.594967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.596604 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.596610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:33.597710 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:33.597722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.598735 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.598742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:33.598928 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:33.599698 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:33.599716 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:33.602455 (+ 2739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":336,"threads_started":1}
W20260927 22:58:33.603060 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:33.593343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:50755 (local address 127.4.74.190:46117)
0927 22:58:33.593680 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:33.593689 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:33.593921 (+ 232us) server_negotiation.cc:407] Connection header received
0927 22:58:33.594045 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:33.594051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:33.594137 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:33.594257 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:33.595153 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.596371 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.597874 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:33.598497 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:33.598668 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:33.602689 (+ 4021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:50755: BlockingRecv error: recv got EOF from 127.4.74.130:50755 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
Sep 27 22:58:34 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:34 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:34.050024 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:34.039327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:56951)
0927 22:58:34.040086 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:34.040107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:34.040296 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:34.040675 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:34.040681 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:34.041186 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:34.041382 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:34.041391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.042716 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.042725 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:34.043954 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:34.043969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.044832 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.044839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:34.045058 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:34.045792 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:34.045812 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:34.049654 (+ 3842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":385,"threads_started":1}
W20260927 22:58:34.050336 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:34.039688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:56951 (local address 127.4.74.190:46117)
0927 22:58:34.039951 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:34.039959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:34.040199 (+ 240us) server_negotiation.cc:407] Connection header received
0927 22:58:34.040433 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:34.040439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:34.040523 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:34.040638 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:34.041531 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.042546 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.044121 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.044646 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.044802 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:34.050019 (+ 5217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:56951: BlockingRecv error: recv got EOF from 127.4.74.129:56951 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Sep 27 22:58:34 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:34 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:34.163633 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:34.153799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:35535)
0927 22:58:34.154517 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:34.154545 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:34.154707 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:34.155059 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:34.155063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:34.155605 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:34.155772 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:34.155781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.156936 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.156943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:34.157887 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:34.157898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.158701 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.158707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:34.158877 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:34.159856 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:34.159872 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:34.163232 (+ 3360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"spinlock_wait_cycles":40704,"thread_start_us":515,"threads_started":1}
W20260927 22:58:34.163789 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:34.153960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:35535 (local address 127.4.74.190:46117)
0927 22:58:34.154306 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:34.154315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:34.154639 (+ 324us) server_negotiation.cc:407] Connection header received
0927 22:58:34.154856 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:34.154860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:34.154926 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:34.155031 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:34.155937 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.156802 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.158025 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.158537 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.158683 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:34.163459 (+ 4776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:35535: BlockingRecv error: recv got EOF from 127.4.74.131:35535 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
Sep 27 22:58:34 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:34 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:34.614070 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:34.604079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:54445)
0927 22:58:34.604774 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:34.604822 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:34.604991 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:34.605391 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:34.605394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:34.605806 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:34.605996 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:34.606004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.607321 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.607325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:34.608319 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:34.608331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.609213 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.609222 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:34.609421 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:34.610193 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:34.610211 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:34.613625 (+ 3414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":353,"threads_started":1}
W20260927 22:58:34.614177 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:34.604209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:54445 (local address 127.4.74.190:46117)
0927 22:58:34.604432 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:34.604440 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:34.604948 (+ 508us) server_negotiation.cc:407] Connection header received
0927 22:58:34.605139 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:34.605143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:34.605230 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:34.605330 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:34.606189 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.607137 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.608484 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:34.608987 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:34.609110 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:34.613891 (+ 4781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:54445: BlockingRecv error: recv got EOF from 127.4.74.130:54445 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 27 22:58:35 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:35 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:35.060555 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:35.051367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:41319)
0927 22:58:35.052020 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:35.052038 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:35.052175 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:35.052624 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:35.052627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:35.053065 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:35.053270 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:35.053280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.054649 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.054653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:35.055613 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:35.055624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.056353 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.056359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:35.056540 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:35.057295 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:35.057326 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:35.060233 (+ 2907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":378,"threads_started":1}
W20260927 22:58:35.061064 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:35.051476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:41319 (local address 127.4.74.190:46117)
0927 22:58:35.051903 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:35.051908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:35.052133 (+ 225us) server_negotiation.cc:407] Connection header received
0927 22:58:35.052309 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:35.052314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:35.052423 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:35.052570 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:35.053479 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.054428 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.055778 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.056205 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.056342 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:35.060721 (+ 4379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:41319: BlockingRecv error: recv got EOF from 127.4.74.129:41319 (error 108)
Metrics: {"server-negotiator.queue_time_us":277}
Sep 27 22:58:35 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:35 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:35.174144 13723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:35.164922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:52897)
0927 22:58:35.165557 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:35.165576 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:35.165788 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:35.166182 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:35.166187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:35.166682 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:35.166898 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:35.166907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.168252 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.168258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:35.169445 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:35.169460 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.170231 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.170236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:35.170360 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:35.171071 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:35.171089 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:35.173893 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":334,"threads_started":1}
W20260927 22:58:35.174492 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:35.165009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:52897 (local address 127.4.74.190:46117)
0927 22:58:35.165315 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:35.165325 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:35.165699 (+ 374us) server_negotiation.cc:407] Connection header received
0927 22:58:35.165961 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:35.165965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:35.166035 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:35.166148 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:35.167055 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.168056 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.169590 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.170053 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.170153 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:35.174151 (+ 3998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:52897: BlockingRecv error: recv got EOF from 127.4.74.131:52897 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 27 22:58:35 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:35 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:35.626284 13724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:35.615427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:34931)
0927 22:58:35.616249 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:35.616270 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:35.616562 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:35.616984 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:35.616989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:35.617544 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:35.617720 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:35.617729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.618891 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.618899 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:35.620294 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:35.620312 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.621437 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.621443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:35.621679 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:35.622464 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:35.622483 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:35.626010 (+ 3527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":498,"threads_started":1}
W20260927 22:58:35.626667 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:35.615787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:34931 (local address 127.4.74.190:46117)
0927 22:58:35.616120 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:35.616128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:35.616450 (+ 322us) server_negotiation.cc:407] Connection header received
0927 22:58:35.616722 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:35.616726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:35.616797 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:35.616922 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:35.617883 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.618722 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.620499 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:35.621234 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:35.621371 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:35.626293 (+ 4922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:34931: BlockingRecv error: recv got EOF from 127.4.74.130:34931 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Sep 27 22:58:36 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:36 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:36.071266 13725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:36.061591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:36441)
0927 22:58:36.062175 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:36.062191 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:36.062307 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:36.062683 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:36.062687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:36.063063 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:36.063276 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:36.063285 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.064618 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.064624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:36.065652 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:36.065667 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.066574 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.066580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:36.066746 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:36.067487 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:36.067506 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:36.070834 (+ 3328us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":292,"threads_started":1}
W20260927 22:58:36.071367 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:36.061674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:36441 (local address 127.4.74.190:46117)
0927 22:58:36.061887 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:36.061892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:36.062279 (+ 387us) server_negotiation.cc:407] Connection header received
0927 22:58:36.062407 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:36.062411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:36.062486 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:36.062582 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:36.063424 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.064463 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.065828 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.066358 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.066483 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:36.071112 (+ 4629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:36441: BlockingRecv error: recv got EOF from 127.4.74.129:36441 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 27 22:58:36 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:36 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:36.185316 13726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:36.175440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:36975)
0927 22:58:36.176255 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:36.176275 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:36.176482 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:36.176899 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:36.176904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:36.177438 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:36.177609 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:36.177618 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.178721 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.178729 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:36.179820 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:36.179837 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.180682 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.180686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:36.180827 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:36.181610 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:36.181629 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:36.184941 (+ 3312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":545,"threads_started":1}
W20260927 22:58:36.185694 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:36.175659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:36975 (local address 127.4.74.190:46117)
0927 22:58:36.175946 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:36.175952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:36.176415 (+ 463us) server_negotiation.cc:407] Connection header received
0927 22:58:36.176629 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:36.176641 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:36.176742 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:36.176859 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:36.177762 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.178567 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.180020 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.180513 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.180649 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:36.185236 (+ 4587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:36975: BlockingRecv error: recv got EOF from 127.4.74.131:36975 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Sep 27 22:58:36 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:36 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:36.637701 13727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:36.627445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:45447)
0927 22:58:36.628425 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:36.628445 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:36.628684 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:36.629101 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:36.629105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:36.629565 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:36.629809 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:36.629820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.630939 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.630944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:36.632067 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:36.632080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.632952 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.632984 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:36.633135 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:36.634009 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:36.634031 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:36.637401 (+ 3370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":859,"thread_start_us":579,"threads_started":1}
W20260927 22:58:36.637995 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:36.627786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:45447 (local address 127.4.74.190:46117)
0927 22:58:36.628001 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:36.628006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:36.628606 (+ 600us) server_negotiation.cc:407] Connection header received
0927 22:58:36.628849 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:36.628854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:36.628932 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:36.629039 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:36.629958 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.630731 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.632254 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:36.632706 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:36.632819 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:36.637713 (+ 4894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:45447: BlockingRecv error: recv got EOF from 127.4.74.130:45447 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 27 22:58:37 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:37 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:37.083223 13728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:37.072719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:34099)
0927 22:58:37.073592 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:37.073619 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:37.073843 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:37.074355 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:37.074360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:37.075008 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:37.075340 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:37.075353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.076553 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.076558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:37.077644 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:37.077659 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.078504 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.078509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:37.078654 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:37.079504 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:37.079524 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:37.082940 (+ 3416us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":350,"threads_started":1}
W20260927 22:58:37.083446 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:37.072969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:34099 (local address 127.4.74.190:46117)
0927 22:58:37.073220 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:37.073227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:37.073771 (+ 544us) server_negotiation.cc:407] Connection header received
0927 22:58:37.074002 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:37.074007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:37.074095 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:37.074220 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:37.075529 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.076365 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.077833 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.078288 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.078398 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:37.083186 (+ 4788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:34099: BlockingRecv error: recv got EOF from 127.4.74.129:34099 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Sep 27 22:58:37 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:37 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:37.196514 13729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:37.186555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:45815)
0927 22:58:37.187288 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:37.187307 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:37.187520 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:37.187984 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:37.187989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:37.188484 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:37.188660 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:37.188669 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.189827 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.189859 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:37.191396 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:37.191413 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.192351 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.192356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:37.192482 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:37.193313 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:37.193332 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:37.196234 (+ 2902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":442,"threads_started":1}
W20260927 22:58:37.196887 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:37.186692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:45815 (local address 127.4.74.190:46117)
0927 22:58:37.187059 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:37.187069 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:37.187430 (+ 361us) server_negotiation.cc:407] Connection header received
0927 22:58:37.187757 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:37.187762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:37.187831 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:37.187951 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:37.188843 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.189620 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.191652 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.192163 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.192294 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:37.196541 (+ 4247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:45815: BlockingRecv error: recv got EOF from 127.4.74.131:45815 (error 108)
Metrics: {"server-negotiator.queue_time_us":180}
Sep 27 22:58:37 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:37 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:37.655376 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:37.639109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:56137)
0927 22:58:37.639807 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:37.639827 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:37.639993 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:37.640392 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:37.640396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:37.640820 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:37.641019 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:37.641029 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.642340 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.642346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:37.643620 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:37.643634 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.644477 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.644483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:37.644675 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:37.651500 (+ 6825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:37.651525 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:37.654990 (+ 3465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":308,"threads_started":1}
W20260927 22:58:37.655758 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:37.639362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:56137 (local address 127.4.74.190:46117)
0927 22:58:37.639680 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:37.639689 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:37.639943 (+ 254us) server_negotiation.cc:407] Connection header received
0927 22:58:37.640127 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:37.640132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:37.640216 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:37.640324 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:37.641198 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.642149 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.643802 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:37.644311 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:37.644461 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:37.655331 (+ 10870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:56137: BlockingRecv error: recv got EOF from 127.4.74.130:56137 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Sep 27 22:58:38 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:38 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.129@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:38.095089 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:38.084976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.129:47889)
0927 22:58:38.085613 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:38.085630 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:38.085796 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:38.086188 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:38.086191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:38.086601 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:38.086843 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:38.086853 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.088026 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.088030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:38.089475 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:38.089490 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.090346 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.090353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:38.090564 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:38.091368 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:38.091389 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:38.094802 (+ 3413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":360,"threads_started":1}
W20260927 22:58:38.095530 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:38.085155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:47889 (local address 127.4.74.190:46117)
0927 22:58:38.085407 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:38.085412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:38.085747 (+ 335us) server_negotiation.cc:407] Connection header received
0927 22:58:38.085920 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:38.085925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:38.086002 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:38.086136 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:38.086997 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.087880 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.089662 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.090123 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.090300 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:38.095139 (+ 4839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.129:47889: BlockingRecv error: recv got EOF from 127.4.74.129:47889 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 27 22:58:38 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:38 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.131@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260927 22:58:38.208178 13732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:38.197592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.131:49505)
0927 22:58:38.198271 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:38.198292 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:38.198431 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:38.198874 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:38.198877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:38.199284 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:38.199504 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:38.199524 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.200823 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.200828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:38.201833 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:38.201845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.202602 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.202607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:38.202762 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:38.203473 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:38.203495 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:38.207852 (+ 4357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":347,"threads_started":1}
W20260927 22:58:38.208379 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:38.197833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:49505 (local address 127.4.74.190:46117)
0927 22:58:38.198032 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:38.198037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:38.198399 (+ 362us) server_negotiation.cc:407] Connection header received
0927 22:58:38.198595 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:38.198600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:38.198694 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:38.198824 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:38.199795 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.200673 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.201988 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.202416 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.202524 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:38.208106 (+ 5582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.131:49505: BlockingRecv error: recv got EOF from 127.4.74.131:49505 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 27 22:58:38 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
Sep 27 22:58:38 dist-test-slave-5pd3 krb5kdc[12682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.74.130@KRBTEST.COM for oryx/127.4.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260927 22:58:38.667016 13554 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:38.657154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:35747 (local address 127.4.74.190:46117)
0927 22:58:38.657390 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:38.657397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:38.657748 (+ 351us) server_negotiation.cc:407] Connection header received
0927 22:58:38.657949 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:38.657957 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:38.658063 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:38.658170 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:38.659473 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.660308 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.661684 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.662125 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.662244 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:38.666813 (+ 4569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.74.130:35747: BlockingRecv error: recv got EOF from 127.4.74.130:35747 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260927 22:58:38.667040 13733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:38.656995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46117 (local address 127.4.74.130:35747)
0927 22:58:38.657609 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:38.657627 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:38.657805 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:38.658238 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:38.658243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:38.658975 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:38.659278 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:38.659292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.660450 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.660454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:38.661543 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:38.661555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:38.662263 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:38.662267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:38.662384 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:38.663126 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:38.663152 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:38.666606 (+ 3454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.74.190:46117: Server oryx/127.4.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":352,"threads_started":1}
I20260927 22:58:39.009249 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13272
I20260927 22:58:39.024564 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:39.010881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36898 (local address 127.4.74.129:42661)
0927 22:58:39.011696 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:39.011705 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:39.011741 (+ 36us) server_negotiation.cc:407] Connection header received
0927 22:58:39.011826 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:39.011834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:39.012145 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:39.012352 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:39.013916 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:39.015081 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:39.016615 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:39.017542 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:39.017691 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:39.020689 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:39.020699 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:39.020720 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:39.020764 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:39.023872 (+ 3108us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.74.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0927 22:58:39.024262 (+ 390us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36898: Request ticket server kudu/127.4.74.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":410,"threads_started":1}
W20260927 22:58:39.024719 13735 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36898: Request ticket server kudu/127.4.74.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260927 22:58:39.024837 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13272: Not authorized: Client connection negotiation failed: client connection to 127.4.74.129:42661: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.74.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260927 22:58:39.025113 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 13272
I20260927 22:58:39.045349 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13423
I20260927 22:58:39.061093 13736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:39.046701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.4.74.130:37297)
0927 22:58:39.047688 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:39.047697 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:39.047731 (+ 34us) server_negotiation.cc:407] Connection header received
0927 22:58:39.047817 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:39.047826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:39.048145 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:39.048367 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:39.049860 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:39.051138 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:39.052955 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:39.053691 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:39.053834 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:39.056770 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:39.056782 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:39.056801 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:39.056856 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:39.060401 (+ 3545us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.74.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0927 22:58:39.060786 (+ 385us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38192: Request ticket server kudu/127.4.74.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":518,"threads_started":1}
W20260927 22:58:39.061245 13736 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38192: Request ticket server kudu/127.4.74.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260927 22:58:39.061324 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13423: Not authorized: Client connection negotiation failed: client connection to 127.4.74.130:37297: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.74.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260927 22:58:39.061579 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 13423
I20260927 22:58:39.079381 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13556
I20260927 22:58:39.093576 13737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:39.080738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44570 (local address 127.4.74.131:42433)
0927 22:58:39.081421 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:39.081427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:39.081456 (+ 29us) server_negotiation.cc:407] Connection header received
0927 22:58:39.081534 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:39.081540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:39.081772 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:39.081951 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:39.083664 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:39.084809 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:39.086441 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:39.087004 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:39.087126 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:39.090148 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:39.090156 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:39.090170 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:39.090235 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:39.093028 (+ 2793us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.74.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0927 22:58:39.093327 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44570: Request ticket server kudu/127.4.74.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":356,"threads_started":1}
W20260927 22:58:39.093670 13737 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44570: Request ticket server kudu/127.4.74.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260927 22:58:39.093997 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13556: Not authorized: Client connection negotiation failed: client connection to 127.4.74.131:42433: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.74.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260927 22:58:39.094194 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 13556
I20260927 22:58:39.113401 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13203
I20260927 22:58:39.115237 13265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:39.254984 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 13203
2026-09-27T22:58:39Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22084 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-27T22:58:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:58:39Z Disabled control of system clock
I20260927 22:58:39.347637 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:46209
--webserver_interface=127.4.74.190
--webserver_port=0
--builtin_ntp_servers=127.4.74.148:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:46209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 22:58:39.783939 13742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:39.784368 13742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:39.784474 13742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:39.792929 13742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:58:39.793084 13742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:39.793125 13742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:39.793181 13742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:58:39.793232 13742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:58:39.803758 13742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:46209
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:46209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:39.805503 13742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:39.808064 13742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:39.824476 13747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:39.824594 13748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:39.824723 13750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:39.826323 13742 server_base.cc:1034] running on GCE node
I20260927 22:58:39.827463 13742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:39.830130 13742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:39.831481 13742 hybrid_clock.cc:648] HybridClock initialized: now 1790549919831399 us; error 75 us; skew 500 ppm
I20260927 22:58:39.832185 13742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:58:39.835393 13742 webserver.cc:460] Webserver started at http://127.4.74.190:34121/ using document root <none> and password file <none>
I20260927 22:58:39.836426 13742 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:39.836584 13742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:39.837010 13742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:39.840847 13742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "d69eeee2407f4c579826638600d02829"
format_stamp: "Formatted at 2026-09-27 22:58:39 on dist-test-slave-5pd3"
server_key: "cc9cdf945d021fbad38745cf00920ecc"
server_key_iv: "2ce38ea236875516a42e7729cfddfc07"
server_key_version: "encryptionkey@0"
I20260927 22:58:39.841871 13742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "d69eeee2407f4c579826638600d02829"
format_stamp: "Formatted at 2026-09-27 22:58:39 on dist-test-slave-5pd3"
server_key: "cc9cdf945d021fbad38745cf00920ecc"
server_key_iv: "2ce38ea236875516a42e7729cfddfc07"
server_key_version: "encryptionkey@0"
I20260927 22:58:39.849123 13742 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260927 22:58:39.854394 13756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:39.856748 13742 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260927 22:58:39.857069 13742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "d69eeee2407f4c579826638600d02829"
format_stamp: "Formatted at 2026-09-27 22:58:39 on dist-test-slave-5pd3"
server_key: "cc9cdf945d021fbad38745cf00920ecc"
server_key_iv: "2ce38ea236875516a42e7729cfddfc07"
server_key_version: "encryptionkey@0"
I20260927 22:58:39.857424 13742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:39.887157 13742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:39.888411 13742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:39.888723 13742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:39.912734 13742 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:46209
I20260927 22:58:39.912762 13807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:46209 every 8 connection(s)
I20260927 22:58:39.914613 13742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:58:39.919272 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 13742
I20260927 22:58:39.919353 13808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:39.919600 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:58:39.920871 4394 external_mini_cluster.cc:1428] Setting key e6b6f5be77283590f9ad6fe52ab824e6
I20260927 22:58:39.933435 13808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829: Bootstrap starting.
I20260927 22:58:39.937512 13808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:39.938865 13808 log.cc:824] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:39.940301 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:39.923922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.4.74.190:46209)
0927 22:58:39.925381 (+ 1459us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:39.925396 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:39.925457 (+ 61us) server_negotiation.cc:407] Connection header received
0927 22:58:39.927507 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:39.927538 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:39.928424 (+ 886us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:39.928918 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:39.929763 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:39.931417 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:39.933743 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:39.934777 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:39.934996 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:39.936657 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:39.936668 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:39.936689 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:39.937435 (+ 746us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:39.937574 (+ 139us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:39.937747 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:39.938014 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:39.938328 (+ 314us) server_negotiation.cc:299] Negotiation successful
0927 22:58:39.938818 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":500,"threads_started":1}
I20260927 22:58:39.942296 13808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829: No bootstrap required, opened a new log
I20260927 22:58:39.950670 13808 raft_consensus.cc:348] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69eeee2407f4c579826638600d02829" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46209 } }
I20260927 22:58:39.951066 13808 raft_consensus.cc:374] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:39.951198 13808 raft_consensus.cc:729] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d69eeee2407f4c579826638600d02829, State: Initialized, Role: FOLLOWER
I20260927 22:58:39.951893 13808 consensus_queue.cc:260] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69eeee2407f4c579826638600d02829" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46209 } }
I20260927 22:58:39.952137 13808 raft_consensus.cc:388] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:58:39.952288 13808 raft_consensus.cc:482] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:58:39.952473 13808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:39.954910 13808 raft_consensus.cc:504] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69eeee2407f4c579826638600d02829" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46209 } }
I20260927 22:58:39.955479 13808 leader_election.cc:302] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d69eeee2407f4c579826638600d02829; no voters:
I20260927 22:58:39.956218 13808 leader_election.cc:288] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:58:39.956544 13813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:39.957427 13813 raft_consensus.cc:686] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [term 1 LEADER]: Becoming Leader. State: Replica: d69eeee2407f4c579826638600d02829, State: Running, Role: LEADER
I20260927 22:58:39.958151 13813 consensus_queue.cc:237] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69eeee2407f4c579826638600d02829" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46209 } }
I20260927 22:58:39.958626 13808 sys_catalog.cc:564] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:58:39.962615 13814 sys_catalog.cc:455] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d69eeee2407f4c579826638600d02829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69eeee2407f4c579826638600d02829" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46209 } } }
I20260927 22:58:39.963266 13814 sys_catalog.cc:458] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:39.962729 13815 sys_catalog.cc:455] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d69eeee2407f4c579826638600d02829. Latest consensus state: current_term: 1 leader_uuid: "d69eeee2407f4c579826638600d02829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69eeee2407f4c579826638600d02829" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46209 } } }
I20260927 22:58:39.963886 13815 sys_catalog.cc:458] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:39.966758 13821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:58:39.973891 13821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:58:39.985286 13821 catalog_manager.cc:1347] Generated new cluster ID: 80083b4b2524490d8a45a01e9140fba0
I20260927 22:58:39.985466 13821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:58:39.998021 13821 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:58:40.002615 13821 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:58:40.022351 13821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d69eeee2407f4c579826638600d02829: Generated new TSK 0
I20260927 22:58:40.024490 13821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 22:58:40.031741 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46209
--builtin_ntp_servers=127.4.74.148:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 22:58:40.448321 13832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:40.448642 13832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:40.448728 13832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:40.456346 13832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:40.456473 13832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:40.456579 13832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:58:40.466668 13832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:40.468125 13832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:40.469993 13832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:40.484871 13838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:40.485069 13837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:40.485235 13840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:40.486763 13832 server_base.cc:1034] running on GCE node
I20260927 22:58:40.487798 13832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:40.489181 13832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:40.490458 13832 hybrid_clock.cc:648] HybridClock initialized: now 1790549920490390 us; error 63 us; skew 500 ppm
I20260927 22:58:40.490924 13832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:58:40.494396 13832 webserver.cc:460] Webserver started at http://127.4.74.129:44339/ using document root <none> and password file <none>
I20260927 22:58:40.495432 13832 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:40.495618 13832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:40.496062 13832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:40.499761 13832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "b68799ed30564dee9a28d0dcd4895aee"
format_stamp: "Formatted at 2026-09-27 22:58:40 on dist-test-slave-5pd3"
server_key: "848b6fed463949d7126ff22f7d06fc0a"
server_key_iv: "cc01a18012e4f097d6689583402a5d8a"
server_key_version: "encryptionkey@0"
I20260927 22:58:40.500774 13832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "b68799ed30564dee9a28d0dcd4895aee"
format_stamp: "Formatted at 2026-09-27 22:58:40 on dist-test-slave-5pd3"
server_key: "848b6fed463949d7126ff22f7d06fc0a"
server_key_iv: "cc01a18012e4f097d6689583402a5d8a"
server_key_version: "encryptionkey@0"
I20260927 22:58:40.507970 13832 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260927 22:58:40.513046 13846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:40.515741 13832 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260927 22:58:40.516023 13832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "b68799ed30564dee9a28d0dcd4895aee"
format_stamp: "Formatted at 2026-09-27 22:58:40 on dist-test-slave-5pd3"
server_key: "848b6fed463949d7126ff22f7d06fc0a"
server_key_iv: "cc01a18012e4f097d6689583402a5d8a"
server_key_version: "encryptionkey@0"
I20260927 22:58:40.516319 13832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:40.532848 13832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:40.533996 13832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:40.534405 13832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:40.535737 13832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:40.537693 13832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:40.537802 13832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:40.537992 13832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:40.538095 13832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:40.580075 13832 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:45589
I20260927 22:58:40.580096 13958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:45589 every 8 connection(s)
I20260927 22:58:40.581938 13832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:58:40.586606 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 13832
I20260927 22:58:40.586858 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:58:40.587829 4394 external_mini_cluster.cc:1428] Setting key aea145c76c1363fd3845d805572cd620
I20260927 22:58:40.592041 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46209
--builtin_ntp_servers=127.4.74.148:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260927 22:58:40.598268 13961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:40.585045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:57397 (local address 127.4.74.190:46209)
0927 22:58:40.585607 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:40.585613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:40.587340 (+ 1727us) server_negotiation.cc:407] Connection header received
0927 22:58:40.590386 (+ 3046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:40.590392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:40.590498 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:40.590671 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:40.592355 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:40.593198 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:40.594971 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:40.595423 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:40.595588 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:40.597329 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:40.597334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:40.597338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:40.597358 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:40.597418 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:40.597431 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:40.597537 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:40.597788 (+ 251us) server_negotiation.cc:299] Negotiation successful
0927 22:58:40.597895 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":307,"threads_started":1}
I20260927 22:58:40.599361 13962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:40.585474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46209 (local address 127.4.74.129:57397)
0927 22:58:40.587028 (+ 1554us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:40.587112 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:40.590012 (+ 2900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:40.591034 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:40.591047 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:40.591171 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:40.592083 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:40.592113 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:40.593364 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:40.593370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:40.594807 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:40.594819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:40.595703 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:40.595711 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:40.595968 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:40.596961 (+ 993us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:58:40.596964 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:40.597007 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:58:40.597092 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:40.597536 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:40.597550 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:40.597560 (+ 10us) client_negotiation.cc:769] Sending connection context
0927 22:58:40.597764 (+ 204us) client_negotiation.cc:240] Negotiation successful
0927 22:58:40.598214 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":442,"threads_started":1}
I20260927 22:58:40.601325 13959 heartbeater.cc:347] Connected to a master server at 127.4.74.190:46209
I20260927 22:58:40.601655 13959 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:40.602450 13959 heartbeater.cc:511] Master 127.4.74.190:46209 requested a full tablet report, sending...
I20260927 22:58:40.605312 13773 ts_manager.cc:194] Registered new tserver with Master: b68799ed30564dee9a28d0dcd4895aee (127.4.74.129:45589)
I20260927 22:58:40.608052 13773 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.129:57397
W20260927 22:58:41.024111 13963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:41.024506 13963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:41.024602 13963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:41.033164 13963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:41.033319 13963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:41.033444 13963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:58:41.044106 13963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:41.045768 13963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:41.047976 13963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:41.064672 13971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:41.064775 13968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:41.064802 13969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:41.066157 13963 server_base.cc:1034] running on GCE node
I20260927 22:58:41.067131 13963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:41.068557 13963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:41.069885 13963 hybrid_clock.cc:648] HybridClock initialized: now 1790549921069801 us; error 70 us; skew 500 ppm
I20260927 22:58:41.070370 13963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:58:41.073684 13963 webserver.cc:460] Webserver started at http://127.4.74.130:32909/ using document root <none> and password file <none>
I20260927 22:58:41.074678 13963 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:41.074834 13963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:41.075240 13963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:41.079268 13963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "2a4b363393f641edb62137d032baa616"
format_stamp: "Formatted at 2026-09-27 22:58:41 on dist-test-slave-5pd3"
server_key: "cfb1ccef53a2800d7a5d0c059862b173"
server_key_iv: "7243fc011d6b5ceda307ba2e2db27d2e"
server_key_version: "encryptionkey@0"
I20260927 22:58:41.080484 13963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "2a4b363393f641edb62137d032baa616"
format_stamp: "Formatted at 2026-09-27 22:58:41 on dist-test-slave-5pd3"
server_key: "cfb1ccef53a2800d7a5d0c059862b173"
server_key_iv: "7243fc011d6b5ceda307ba2e2db27d2e"
server_key_version: "encryptionkey@0"
I20260927 22:58:41.088167 13963 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260927 22:58:41.093267 13977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:41.095535 13963 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260927 22:58:41.095921 13963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "2a4b363393f641edb62137d032baa616"
format_stamp: "Formatted at 2026-09-27 22:58:41 on dist-test-slave-5pd3"
server_key: "cfb1ccef53a2800d7a5d0c059862b173"
server_key_iv: "7243fc011d6b5ceda307ba2e2db27d2e"
server_key_version: "encryptionkey@0"
I20260927 22:58:41.096253 13963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:41.110301 13963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:41.111537 13963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:41.112022 13963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:41.113409 13963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:41.115355 13963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:41.115463 13963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:41.115644 13963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:41.115718 13963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:41.155337 13963 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:43357
I20260927 22:58:41.155412 14089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:43357 every 8 connection(s)
I20260927 22:58:41.157435 13963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:58:41.167284 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 13963
I20260927 22:58:41.167483 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:58:41.168150 4394 external_mini_cluster.cc:1428] Setting key e59be6c57988aa275077262fb2489b59
I20260927 22:58:41.171933 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46209
--builtin_ntp_servers=127.4.74.148:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260927 22:58:41.175635 14092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:41.161375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:33699 (local address 127.4.74.190:46209)
0927 22:58:41.162024 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:41.162029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:41.163892 (+ 1863us) server_negotiation.cc:407] Connection header received
0927 22:58:41.165924 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:41.165930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:41.166094 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:41.166280 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:41.168151 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.169137 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.171527 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.172304 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.172469 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:41.174342 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:41.174352 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:41.174360 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:41.174398 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:41.174507 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:41.174531 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:41.174654 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:41.174943 (+ 289us) server_negotiation.cc:299] Negotiation successful
0927 22:58:41.175089 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":332,"threads_started":1}
I20260927 22:58:41.176614 14093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:41.162226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46209 (local address 127.4.74.130:33699)
0927 22:58:41.163665 (+ 1439us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:41.163714 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:41.165531 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:41.166616 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:41.166626 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:41.166776 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:41.167843 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:41.167874 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.169451 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.169458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:41.171309 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:41.171324 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.172594 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.172600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:41.172846 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:41.173838 (+ 992us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:58:41.173844 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:41.173908 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:58:41.174044 (+ 136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:41.174691 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:41.174699 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:41.174707 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 22:58:41.174899 (+ 192us) client_negotiation.cc:240] Negotiation successful
0927 22:58:41.175284 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":416,"threads_started":1}
I20260927 22:58:41.178524 14090 heartbeater.cc:347] Connected to a master server at 127.4.74.190:46209
I20260927 22:58:41.178905 14090 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:41.179827 14090 heartbeater.cc:511] Master 127.4.74.190:46209 requested a full tablet report, sending...
I20260927 22:58:41.181792 13773 ts_manager.cc:194] Registered new tserver with Master: 2a4b363393f641edb62137d032baa616 (127.4.74.130:43357)
I20260927 22:58:41.183043 13773 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.130:33699
W20260927 22:58:41.598932 14094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:41.599323 14094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:41.599485 14094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:41.608135 14094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:41.608287 14094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:41.608412 14094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:58:41.611951 13959 heartbeater.cc:503] Master 127.4.74.190:46209 was elected leader, sending a full tablet report...
I20260927 22:58:41.619429 14094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:41.621219 14094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:41.623711 14094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:41.640771 14099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:41.640864 14100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:41.640852 14102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:41.642323 14094 server_base.cc:1034] running on GCE node
I20260927 22:58:41.643433 14094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:41.644840 14094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:41.646162 14094 hybrid_clock.cc:648] HybridClock initialized: now 1790549921646086 us; error 100 us; skew 500 ppm
I20260927 22:58:41.646701 14094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:58:41.649646 14094 webserver.cc:460] Webserver started at http://127.4.74.131:41091/ using document root <none> and password file <none>
I20260927 22:58:41.650527 14094 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:41.650701 14094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:41.651072 14094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:41.654453 14094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "26ab1cb77b8348318e85471290601dc1"
format_stamp: "Formatted at 2026-09-27 22:58:41 on dist-test-slave-5pd3"
server_key: "45446e156165fc079f5026c2fab026b6"
server_key_iv: "b748316391bec1c2854136d9d72d32d2"
server_key_version: "encryptionkey@0"
I20260927 22:58:41.655349 14094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "26ab1cb77b8348318e85471290601dc1"
format_stamp: "Formatted at 2026-09-27 22:58:41 on dist-test-slave-5pd3"
server_key: "45446e156165fc079f5026c2fab026b6"
server_key_iv: "b748316391bec1c2854136d9d72d32d2"
server_key_version: "encryptionkey@0"
I20260927 22:58:41.662118 14094 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260927 22:58:41.668005 14108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:41.670468 14094 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260927 22:58:41.670787 14094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "26ab1cb77b8348318e85471290601dc1"
format_stamp: "Formatted at 2026-09-27 22:58:41 on dist-test-slave-5pd3"
server_key: "45446e156165fc079f5026c2fab026b6"
server_key_iv: "b748316391bec1c2854136d9d72d32d2"
server_key_version: "encryptionkey@0"
I20260927 22:58:41.671084 14094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:41.687013 14094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:41.688066 14094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:41.688413 14094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:41.689633 14094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:41.691371 14094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:41.691507 14094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:41.691684 14094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:41.691776 14094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:41.733760 14094 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:36325
I20260927 22:58:41.734050 14220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:36325 every 8 connection(s)
I20260927 22:58:41.735285 14094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:58:41.736081 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 14094
I20260927 22:58:41.736313 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:58:41.736985 4394 external_mini_cluster.cc:1428] Setting key 6f6e443f4b4fd62db57a0ce8d09a0c9c
I20260927 22:58:41.750833 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:41.738440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:58425 (local address 127.4.74.190:46209)
0927 22:58:41.739055 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:41.739060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:41.741220 (+ 2160us) server_negotiation.cc:407] Connection header received
0927 22:58:41.743299 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:41.743303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:41.743396 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:41.743510 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:41.744940 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.745767 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.747710 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.748125 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.748230 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:41.749964 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:41.749969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:41.749972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:41.749994 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:41.750043 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:41.750055 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:41.750131 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:41.750415 (+ 284us) server_negotiation.cc:299] Negotiation successful
0927 22:58:41.750502 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":315,"threads_started":1}
I20260927 22:58:41.752231 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:41.739673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46209 (local address 127.4.74.131:58425)
0927 22:58:41.741024 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:41.741077 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:41.742930 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:41.743856 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:41.743867 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:41.743988 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:41.744731 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:41.744749 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.745961 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.745966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:41.747473 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:41.747487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.748355 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.748360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:41.748627 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:41.749558 (+ 931us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:58:41.749561 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:41.749625 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:58:41.749726 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:41.750159 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:41.750167 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:41.750174 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 22:58:41.750358 (+ 184us) client_negotiation.cc:240] Negotiation successful
0927 22:58:41.750769 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":421,"threads_started":1}
I20260927 22:58:41.754089 14221 heartbeater.cc:347] Connected to a master server at 127.4.74.190:46209
I20260927 22:58:41.754470 14221 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:41.755787 14221 heartbeater.cc:511] Master 127.4.74.190:46209 requested a full tablet report, sending...
I20260927 22:58:41.757588 13773 ts_manager.cc:194] Registered new tserver with Master: 26ab1cb77b8348318e85471290601dc1 (127.4.74.131:36325)
I20260927 22:58:41.758723 13773 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.131:58425
I20260927 22:58:41.766813 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 22:58:41.774206 14223 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:41.772271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.4.74.190:46209)
0927 22:58:41.772487 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:41.772495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:41.773091 (+ 596us) server_negotiation.cc:407] Connection header received
0927 22:58:41.773314 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:41.773320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:41.773395 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:41.773511 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:41.773907 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52160: BlockingRecv error: recv got EOF from 127.0.0.1:52160 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 22:58:41.775389 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13832
I20260927 22:58:41.784428 14231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:41.776541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42854 (local address 127.4.74.129:45589)
0927 22:58:41.777181 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:41.777189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:41.777219 (+ 30us) server_negotiation.cc:407] Connection header received
0927 22:58:41.777278 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:41.777285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:41.777510 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:41.777651 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:41.778121 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.779176 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.780649 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.781297 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.781413 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:41.782822 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:41.782831 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:41.782845 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:41.783466 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:41.783515 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:41.783535 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:41.783737 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:41.783893 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 22:58:41.784014 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":273,"threads_started":1}
I20260927 22:58:41.786098 13954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:41.874439 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 13832
I20260927 22:58:41.892352 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13963
I20260927 22:58:41.901450 14233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:41.893435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.4.74.130:43357)
0927 22:58:41.894015 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:41.894023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:41.894055 (+ 32us) server_negotiation.cc:407] Connection header received
0927 22:58:41.894125 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:41.894131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:41.894355 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:41.894503 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:41.895008 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.896191 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.897623 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:41.898214 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:41.898324 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:41.899738 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:41.899746 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:41.899759 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:41.900465 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:41.900519 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:41.900539 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:41.900709 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:41.900903 (+ 194us) server_negotiation.cc:299] Negotiation successful
0927 22:58:41.901021 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":299,"threads_started":1}
I20260927 22:58:41.902988 14085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:41.989692 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 13963
I20260927 22:58:42.007999 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 14094
I20260927 22:58:42.018483 14235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:42.009415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60856 (local address 127.4.74.131:36325)
0927 22:58:42.010200 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:42.010209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:42.010245 (+ 36us) server_negotiation.cc:407] Connection header received
0927 22:58:42.010325 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:42.010333 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:42.010635 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:42.010816 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:42.011444 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:42.012800 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:42.014165 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:42.014850 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:42.014965 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:42.016415 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:42.016425 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:42.016446 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:42.017279 (+ 833us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:42.017349 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:42.017382 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:42.017510 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:42.017881 (+ 371us) server_negotiation.cc:299] Negotiation successful
0927 22:58:42.018069 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":373,"threads_started":1}
I20260927 22:58:42.020236 14216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:42.117597 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 14094
I20260927 22:58:42.134374 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 13742
I20260927 22:58:42.135865 13803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:42.242303 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 13742
2026-09-27T22:58:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2982 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-27T22:58:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:58:42Z Disabled control of system clock
I20260927 22:58:42.331139 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:34085
--webserver_interface=127.4.74.190
--webserver_port=0
--builtin_ntp_servers=127.4.74.148:38159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:34085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 22:58:42.760094 14241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:42.760504 14241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:42.760679 14241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:42.769673 14241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:58:42.769894 14241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:42.769994 14241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:42.770116 14241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:58:42.770227 14241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:58:42.782271 14241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:34085
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:34085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:42.784161 14241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:42.786536 14241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:42.802037 14247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:42.802007 14249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:42.802096 14246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:42.805799 14241 server_base.cc:1034] running on GCE node
I20260927 22:58:42.806942 14241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:42.808769 14241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:42.810065 14241 hybrid_clock.cc:648] HybridClock initialized: now 1790549922810004 us; error 62 us; skew 500 ppm
I20260927 22:58:42.810541 14241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:58:42.818236 14241 webserver.cc:460] Webserver started at http://127.4.74.190:38989/ using document root <none> and password file <none>
I20260927 22:58:42.819169 14241 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:42.819275 14241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:42.819737 14241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:42.823017 14241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "493dac7d9c1240c28ecf3b20c9339a46"
format_stamp: "Formatted at 2026-09-27 22:58:42 on dist-test-slave-5pd3"
server_key: "d1f9913f5b6babaa31e5a6f29a56a536"
server_key_iv: "74d3c5d40ce28752685c9bcb765712d6"
server_key_version: "encryptionkey@0"
I20260927 22:58:42.823977 14241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "493dac7d9c1240c28ecf3b20c9339a46"
format_stamp: "Formatted at 2026-09-27 22:58:42 on dist-test-slave-5pd3"
server_key: "d1f9913f5b6babaa31e5a6f29a56a536"
server_key_iv: "74d3c5d40ce28752685c9bcb765712d6"
server_key_version: "encryptionkey@0"
I20260927 22:58:42.830188 14241 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 22:58:42.834827 14255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:42.836790 14241 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260927 22:58:42.837045 14241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "493dac7d9c1240c28ecf3b20c9339a46"
format_stamp: "Formatted at 2026-09-27 22:58:42 on dist-test-slave-5pd3"
server_key: "d1f9913f5b6babaa31e5a6f29a56a536"
server_key_iv: "74d3c5d40ce28752685c9bcb765712d6"
server_key_version: "encryptionkey@0"
I20260927 22:58:42.837293 14241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:42.844765 14241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:42.845741 14241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:42.846189 14241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:42.868530 14241 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:34085
I20260927 22:58:42.868638 14306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:34085 every 8 connection(s)
I20260927 22:58:42.870095 14241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:58:42.873580 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 14241
I20260927 22:58:42.873831 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:58:42.874518 4394 external_mini_cluster.cc:1428] Setting key fbd3bb15714181801bcf8cd8b07c8f1c
I20260927 22:58:42.875433 14307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:42.885772 14310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:42.877023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42088 (local address 127.4.74.190:34085)
0927 22:58:42.878212 (+ 1189us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:42.878224 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:42.878290 (+ 66us) server_negotiation.cc:407] Connection header received
0927 22:58:42.881206 (+ 2916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:42.881235 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:42.881822 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:42.882237 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:42.883798 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:42.883805 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:42.883819 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:42.883837 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:42.883917 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:42.883939 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:42.884208 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:42.884354 (+ 146us) server_negotiation.cc:299] Negotiation successful
0927 22:58:42.884718 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":436,"threads_started":1}
I20260927 22:58:42.892499 14307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46: Bootstrap starting.
I20260927 22:58:42.896600 14307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:42.898047 14307 log.cc:824] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:42.902070 14307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46: No bootstrap required, opened a new log
I20260927 22:58:42.910116 14307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493dac7d9c1240c28ecf3b20c9339a46" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34085 } }
I20260927 22:58:42.910627 14307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:42.910769 14307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 493dac7d9c1240c28ecf3b20c9339a46, State: Initialized, Role: FOLLOWER
I20260927 22:58:42.911486 14307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493dac7d9c1240c28ecf3b20c9339a46" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34085 } }
I20260927 22:58:42.911813 14307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:58:42.911996 14307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:58:42.912177 14307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:42.914608 14307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493dac7d9c1240c28ecf3b20c9339a46" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34085 } }
I20260927 22:58:42.915215 14307 leader_election.cc:302] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 493dac7d9c1240c28ecf3b20c9339a46; no voters:
I20260927 22:58:42.916023 14307 leader_election.cc:288] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:58:42.916333 14312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:42.917371 14312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [term 1 LEADER]: Becoming Leader. State: Replica: 493dac7d9c1240c28ecf3b20c9339a46, State: Running, Role: LEADER
I20260927 22:58:42.918135 14312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493dac7d9c1240c28ecf3b20c9339a46" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34085 } }
I20260927 22:58:42.918447 14307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:58:42.921857 14313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "493dac7d9c1240c28ecf3b20c9339a46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493dac7d9c1240c28ecf3b20c9339a46" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34085 } } }
I20260927 22:58:42.922065 14314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 493dac7d9c1240c28ecf3b20c9339a46. Latest consensus state: current_term: 1 leader_uuid: "493dac7d9c1240c28ecf3b20c9339a46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "493dac7d9c1240c28ecf3b20c9339a46" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34085 } } }
I20260927 22:58:42.922538 14313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:42.922538 14314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46 [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:42.923698 14319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:58:42.930553 14319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:58:42.942669 14319 catalog_manager.cc:1347] Generated new cluster ID: 2b9f084d5f8e41959194762df48bbf1c
I20260927 22:58:42.942940 14319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:58:42.965658 14319 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:58:42.966219 14319 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:58:42.986344 14319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 493dac7d9c1240c28ecf3b20c9339a46: Generated new TSK 0
I20260927 22:58:42.987697 14319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 22:58:43.006314 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34085
--builtin_ntp_servers=127.4.74.148:38159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 22:58:43.409739 14331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:43.410156 14331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:43.410338 14331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:43.419217 14331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:43.419487 14331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:43.419808 14331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:58:43.430751 14331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:43.432622 14331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:43.434892 14331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:43.452822 14336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:43.452793 14337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:43.453567 14339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:43.454007 14331 server_base.cc:1034] running on GCE node
I20260927 22:58:43.454820 14331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:43.456045 14331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:43.457324 14331 hybrid_clock.cc:648] HybridClock initialized: now 1790549923457241 us; error 80 us; skew 500 ppm
I20260927 22:58:43.457744 14331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:58:43.460472 14331 webserver.cc:460] Webserver started at http://127.4.74.129:33673/ using document root <none> and password file <none>
I20260927 22:58:43.461326 14331 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:43.461460 14331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:43.461839 14331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:43.465077 14331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "b80e6268dd6f4d9ba289c443340ef0e9"
format_stamp: "Formatted at 2026-09-27 22:58:43 on dist-test-slave-5pd3"
server_key: "fd423e63f4fdae423d555db9258ad83e"
server_key_iv: "dfc5491d725196d91f4fa6962ccf5fd7"
server_key_version: "encryptionkey@0"
I20260927 22:58:43.465952 14331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "b80e6268dd6f4d9ba289c443340ef0e9"
format_stamp: "Formatted at 2026-09-27 22:58:43 on dist-test-slave-5pd3"
server_key: "fd423e63f4fdae423d555db9258ad83e"
server_key_iv: "dfc5491d725196d91f4fa6962ccf5fd7"
server_key_version: "encryptionkey@0"
I20260927 22:58:43.471948 14331 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 22:58:43.476127 14345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:43.477885 14331 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 22:58:43.478111 14331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "b80e6268dd6f4d9ba289c443340ef0e9"
format_stamp: "Formatted at 2026-09-27 22:58:43 on dist-test-slave-5pd3"
server_key: "fd423e63f4fdae423d555db9258ad83e"
server_key_iv: "dfc5491d725196d91f4fa6962ccf5fd7"
server_key_version: "encryptionkey@0"
I20260927 22:58:43.478359 14331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:43.484666 14331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:43.485538 14331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:43.485833 14331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:43.486934 14331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:43.488603 14331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:43.488745 14331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:43.488875 14331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:43.488961 14331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:43.528789 14331 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:40775
I20260927 22:58:43.529026 14457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:40775 every 8 connection(s)
I20260927 22:58:43.530236 14331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:58:43.534538 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 14331
I20260927 22:58:43.534771 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:58:43.535650 4394 external_mini_cluster.cc:1428] Setting key d7681449ded78468177f77930fa0f214
I20260927 22:58:43.538651 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34085
--builtin_ntp_servers=127.4.74.148:38159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260927 22:58:43.540659 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:43.533554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:57269 (local address 127.4.74.190:34085)
0927 22:58:43.534137 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:43.534142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:43.535567 (+ 1425us) server_negotiation.cc:407] Connection header received
0927 22:58:43.537846 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:43.537852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:43.537930 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:43.538072 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:43.539732 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:43.539738 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:43.539743 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:43.539769 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:43.539818 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:43.539831 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:43.539989 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:43.540209 (+ 220us) server_negotiation.cc:299] Negotiation successful
0927 22:58:43.540290 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":323,"threads_started":1}
I20260927 22:58:43.541970 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:43.533946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34085 (local address 127.4.74.129:57269)
0927 22:58:43.535331 (+ 1385us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:43.535380 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:43.537508 (+ 2128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:43.538373 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:43.538383 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:43.538505 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:43.539348 (+ 843us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:58:43.539350 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:43.539396 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:58:43.539501 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:43.539950 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:43.539963 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:43.539973 (+ 10us) client_negotiation.cc:769] Sending connection context
0927 22:58:43.540299 (+ 326us) client_negotiation.cc:240] Negotiation successful
0927 22:58:43.540630 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":374,"threads_started":1}
I20260927 22:58:43.544170 14458 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34085
I20260927 22:58:43.544549 14458 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:43.545274 14458 heartbeater.cc:511] Master 127.4.74.190:34085 requested a full tablet report, sending...
I20260927 22:58:43.548034 14272 ts_manager.cc:194] Registered new tserver with Master: b80e6268dd6f4d9ba289c443340ef0e9 (127.4.74.129:40775)
W20260927 22:58:43.961683 14462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:43.962430 14462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:43.962628 14462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:43.971153 14462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:43.971371 14462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:43.971591 14462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:58:43.982918 14462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:43.984740 14462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:43.987144 14462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:44.005620 14467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:44.005635 14468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:44.005548 14470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:44.006517 14462 server_base.cc:1034] running on GCE node
I20260927 22:58:44.007334 14462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:44.008651 14462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:44.009964 14462 hybrid_clock.cc:648] HybridClock initialized: now 1790549924009866 us; error 41 us; skew 500 ppm
I20260927 22:58:44.010434 14462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:58:44.013648 14462 webserver.cc:460] Webserver started at http://127.4.74.130:44793/ using document root <none> and password file <none>
I20260927 22:58:44.014693 14462 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:44.014822 14462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:44.015295 14462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:44.019299 14462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "f19f4736e3b14c85a4d8164cb73885b1"
format_stamp: "Formatted at 2026-09-27 22:58:44 on dist-test-slave-5pd3"
server_key: "0121732366d114e7097f5596a39b1f90"
server_key_iv: "1d1308640820b5c49aa375628b5a664e"
server_key_version: "encryptionkey@0"
I20260927 22:58:44.020558 14462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "f19f4736e3b14c85a4d8164cb73885b1"
format_stamp: "Formatted at 2026-09-27 22:58:44 on dist-test-slave-5pd3"
server_key: "0121732366d114e7097f5596a39b1f90"
server_key_iv: "1d1308640820b5c49aa375628b5a664e"
server_key_version: "encryptionkey@0"
I20260927 22:58:44.028434 14462 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260927 22:58:44.033535 14476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:44.035635 14462 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260927 22:58:44.036123 14462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "f19f4736e3b14c85a4d8164cb73885b1"
format_stamp: "Formatted at 2026-09-27 22:58:44 on dist-test-slave-5pd3"
server_key: "0121732366d114e7097f5596a39b1f90"
server_key_iv: "1d1308640820b5c49aa375628b5a664e"
server_key_version: "encryptionkey@0"
I20260927 22:58:44.036478 14462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:44.044315 14462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:44.045485 14462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:44.045871 14462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:44.047082 14462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:44.048954 14462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:44.049054 14462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:44.049163 14462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:44.049232 14462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:44.091938 14462 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:33209
I20260927 22:58:44.092001 14588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:33209 every 8 connection(s)
I20260927 22:58:44.093715 14462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:58:44.099511 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 14462
I20260927 22:58:44.099752 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:58:44.100330 4394 external_mini_cluster.cc:1428] Setting key 2b0b59094cfb3ecd23557fbc89b135ba
I20260927 22:58:44.103829 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34085
--builtin_ntp_servers=127.4.74.148:38159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260927 22:58:44.106984 14592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:44.097675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34085 (local address 127.4.74.130:37705)
0927 22:58:44.098922 (+ 1247us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:44.099014 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:44.101447 (+ 2433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:44.103008 (+ 1561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:44.103020 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:44.103161 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:44.104239 (+ 1078us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:58:44.104245 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:44.104301 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:58:44.104428 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:44.105014 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:44.105028 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:44.105038 (+ 10us) client_negotiation.cc:769] Sending connection context
0927 22:58:44.105334 (+ 296us) client_negotiation.cc:240] Negotiation successful
0927 22:58:44.105650 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":347,"threads_started":1}
I20260927 22:58:44.107687 14591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:44.097058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:37705 (local address 127.4.74.190:34085)
0927 22:58:44.097570 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:44.097574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:44.099244 (+ 1670us) server_negotiation.cc:407] Connection header received
0927 22:58:44.101883 (+ 2639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:44.101894 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:44.102460 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:44.102665 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:44.104679 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:44.104688 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:44.104696 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:44.104725 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:44.104791 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:44.104826 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:44.104959 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:44.107214 (+ 2255us) server_negotiation.cc:299] Negotiation successful
0927 22:58:44.107292 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":274,"threads_started":1}
I20260927 22:58:44.116140 14589 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34085
I20260927 22:58:44.116626 14589 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:44.117619 14589 heartbeater.cc:511] Master 127.4.74.190:34085 requested a full tablet report, sending...
I20260927 22:58:44.120083 14272 ts_manager.cc:194] Registered new tserver with Master: f19f4736e3b14c85a4d8164cb73885b1 (127.4.74.130:33209)
W20260927 22:58:44.487367 14593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:44.488436 14593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:44.488595 14593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:44.496924 14593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:44.497052 14593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:44.497244 14593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:58:44.506978 14593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:44.508507 14593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:44.510505 14593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:44.526535 14601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:44.526674 14599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:44.526554 14598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:44.529958 14593 server_base.cc:1034] running on GCE node
I20260927 22:58:44.530746 14593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:44.532035 14593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:44.533267 14593 hybrid_clock.cc:648] HybridClock initialized: now 1790549924533211 us; error 64 us; skew 500 ppm
I20260927 22:58:44.533615 14593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 22:58:44.536427 14593 webserver.cc:460] Webserver started at http://127.4.74.131:37211/ using document root <none> and password file <none>
I20260927 22:58:44.537158 14593 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:44.537256 14593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:44.537587 14593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:44.540756 14593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "e8efe0076bd84ea2bb223297b2cca222"
format_stamp: "Formatted at 2026-09-27 22:58:44 on dist-test-slave-5pd3"
server_key: "8e4ecd5c04bba2f8b45f1697cd2bcab7"
server_key_iv: "5f5f6a7082809e97ed0da5006b3a77bc"
server_key_version: "encryptionkey@0"
I20260927 22:58:44.541664 14593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "e8efe0076bd84ea2bb223297b2cca222"
format_stamp: "Formatted at 2026-09-27 22:58:44 on dist-test-slave-5pd3"
server_key: "8e4ecd5c04bba2f8b45f1697cd2bcab7"
server_key_iv: "5f5f6a7082809e97ed0da5006b3a77bc"
server_key_version: "encryptionkey@0"
I20260927 22:58:44.547513 14593 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260927 22:58:44.551644 14607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:44.551831 14458 heartbeater.cc:503] Master 127.4.74.190:34085 was elected leader, sending a full tablet report...
I20260927 22:58:44.553495 14593 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:58:44.553696 14593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "e8efe0076bd84ea2bb223297b2cca222"
format_stamp: "Formatted at 2026-09-27 22:58:44 on dist-test-slave-5pd3"
server_key: "8e4ecd5c04bba2f8b45f1697cd2bcab7"
server_key_iv: "5f5f6a7082809e97ed0da5006b3a77bc"
server_key_version: "encryptionkey@0"
I20260927 22:58:44.554026 14593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:44.560410 14593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:44.561252 14593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:44.561501 14593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:44.562475 14593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:44.564061 14593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:44.564157 14593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:44.564249 14593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:44.564311 14593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:44.604702 14593 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:33593
I20260927 22:58:44.604813 14719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:33593 every 8 connection(s)
I20260927 22:58:44.606143 14593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:58:44.610145 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 14593
I20260927 22:58:44.610414 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:58:44.610999 4394 external_mini_cluster.cc:1428] Setting key a464e7762e9188d29e753cbde701e09d
I20260927 22:58:44.616564 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:44.610041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:58235 (local address 127.4.74.190:34085)
0927 22:58:44.610665 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:44.610674 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:44.612045 (+ 1371us) server_negotiation.cc:407] Connection header received
0927 22:58:44.614058 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:44.614063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:44.614136 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:44.614258 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:44.615746 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:44.615750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:44.615755 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:44.615774 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:44.615831 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:44.615845 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:44.615958 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:44.616181 (+ 223us) server_negotiation.cc:299] Negotiation successful
0927 22:58:44.616249 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":281,"threads_started":1}
I20260927 22:58:44.617369 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:44.610617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34085 (local address 127.4.74.131:58235)
0927 22:58:44.611857 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:44.611904 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:44.613705 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:44.614527 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:44.614535 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:44.614633 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:44.615388 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 22:58:44.615391 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:44.615430 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 22:58:44.615515 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:44.615948 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:44.615956 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:44.615961 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 22:58:44.616184 (+ 223us) client_negotiation.cc:240] Negotiation successful
0927 22:58:44.616383 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":419,"threads_started":1}
I20260927 22:58:44.619216 14720 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34085
I20260927 22:58:44.619592 14720 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:44.620294 14720 heartbeater.cc:511] Master 127.4.74.190:34085 requested a full tablet report, sending...
I20260927 22:58:44.621984 14272 ts_manager.cc:194] Registered new tserver with Master: e8efe0076bd84ea2bb223297b2cca222 (127.4.74.131:33593)
I20260927 22:58:44.628024 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 22:58:44.635531 14722 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 22:58:44.633743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42092 (local address 127.4.74.190:34085)
0927 22:58:44.633944 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:44.633948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:44.634433 (+ 485us) server_negotiation.cc:407] Connection header received
0927 22:58:44.634607 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:44.634611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:44.634700 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:44.634836 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:44.635310 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42092: BlockingRecv error: recv got EOF from 127.0.0.1:42092 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 22:58:44.637684 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 14331
I20260927 22:58:44.643400 14730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:44.638858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56948 (local address 127.4.74.129:40775)
0927 22:58:44.639673 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:44.639681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:44.639714 (+ 33us) server_negotiation.cc:407] Connection header received
0927 22:58:44.639792 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:44.639799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:44.640036 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:44.640204 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:44.641885 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:44.641896 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:44.641908 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:44.641921 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:44.641965 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:44.641980 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:44.642292 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:44.642913 (+ 621us) server_negotiation.cc:299] Negotiation successful
0927 22:58:44.643017 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":303,"threads_started":1}
I20260927 22:58:44.644920 14453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:44.730636 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 14331
I20260927 22:58:44.750427 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 14462
I20260927 22:58:44.754701 14732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:44.751692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.4.74.130:33209)
0927 22:58:44.752270 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:44.752277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:44.752308 (+ 31us) server_negotiation.cc:407] Connection header received
0927 22:58:44.752380 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:44.752387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:44.752626 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:44.752780 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:44.753838 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:44.753848 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:44.753859 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:44.753880 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:44.753924 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:44.753937 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:44.754052 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:44.754254 (+ 202us) server_negotiation.cc:299] Negotiation successful
0927 22:58:44.754335 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":287,"threads_started":1}
I20260927 22:58:44.756273 14584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:44.844147 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 14462
I20260927 22:58:44.862671 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 14593
I20260927 22:58:44.867710 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:44.864030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50022 (local address 127.4.74.131:33593)
0927 22:58:44.864759 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:44.864768 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:44.864804 (+ 36us) server_negotiation.cc:407] Connection header received
0927 22:58:44.864882 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:44.864890 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:44.865186 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:44.865397 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:44.866735 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:44.866742 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:44.866758 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 22:58:44.866787 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:44.866834 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 22:58:44.866850 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:44.867071 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:44.867154 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 22:58:44.867274 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":357,"threads_started":1}
I20260927 22:58:44.869309 14715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:44.961251 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 14593
I20260927 22:58:44.979183 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 14241
I20260927 22:58:44.980706 14302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:45.085760 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 14241
2026-09-27T22:58:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2844 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:58:45 dist-test-slave-5pd3 krb5kdc[14739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:58:45 dist-test-slave-5pd3 krb5kdc[14739](info): set up 2 sockets
Sep 27 22:58:45 dist-test-slave-5pd3 krb5kdc[14739](info): commencing operation
krb5kdc: starting...
W20260927 22:58:46.228806 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:58:46 dist-test-slave-5pd3 krb5kdc[14739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:58:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:58:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:58:46.512868 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36669
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:46.856146 14752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:46.856454 14752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:46.856613 14752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:46.863941 14752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:58:46.864068 14752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:46.864113 14752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:46.864239 14752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:58:46.864338 14752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:58:46.874347 14752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:46.875787 14752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:46.877149 14752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:46.890519 14757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:46.890511 14760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:46.890733 14758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:46.891570 14752 server_base.cc:1034] running on GCE node
I20260927 22:58:46.892447 14752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:46.894050 14752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:46.895310 14752 hybrid_clock.cc:648] HybridClock initialized: now 1790549926895240 us; error 65 us; skew 500 ppm
I20260927 22:58:46.895753 14752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:46 dist-test-slave-5pd3 krb5kdc[14739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549926, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:46.899161 14752 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:58:46.901049 14752 webserver.cc:460] Webserver started at http://127.4.74.190:37105/ using document root <none> and password file <none>
I20260927 22:58:46.901782 14752 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:46.901940 14752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:46.902257 14752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:46.905395 14752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "70f48f09f3fd4039bdd0784b7011b02d"
format_stamp: "Formatted at 2026-09-27 22:58:46 on dist-test-slave-5pd3"
server_key: "f5c3973cb1242f6822e375ec0b462cc6"
server_key_iv: "4f6c1a74e0591db1f320f3cad71946db"
server_key_version: "encryptionkey@0"
I20260927 22:58:46.906239 14752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "70f48f09f3fd4039bdd0784b7011b02d"
format_stamp: "Formatted at 2026-09-27 22:58:46 on dist-test-slave-5pd3"
server_key: "f5c3973cb1242f6822e375ec0b462cc6"
server_key_iv: "4f6c1a74e0591db1f320f3cad71946db"
server_key_version: "encryptionkey@0"
I20260927 22:58:46.912278 14752 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260927 22:58:46.916282 14767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:46.918143 14752 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 22:58:46.918390 14752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "70f48f09f3fd4039bdd0784b7011b02d"
format_stamp: "Formatted at 2026-09-27 22:58:46 on dist-test-slave-5pd3"
server_key: "f5c3973cb1242f6822e375ec0b462cc6"
server_key_iv: "4f6c1a74e0591db1f320f3cad71946db"
server_key_version: "encryptionkey@0"
I20260927 22:58:46.918618 14752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:46.935640 14752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:46.940352 14752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:46.940675 14752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:46.962440 14752 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:36669
I20260927 22:58:46.962522 14818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:36669 every 8 connection(s)
I20260927 22:58:46.963943 14752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:58:46.968772 14819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:46.972371 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 14752
I20260927 22:58:46.972682 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:58:46.973572 4394 external_mini_cluster.cc:1428] Setting key dfe9bd169b0e054208c95fc6216c06ec
I20260927 22:58:46.984862 14819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d: Bootstrap starting.
I20260927 22:58:46.989964 14819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:46.991760 14819 log.cc:824] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d: Log is configured to *not* fsync() on all Append() calls
Sep 27 22:58:46 dist-test-slave-5pd3 krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:58:46.995672 14819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d: No bootstrap required, opened a new log
I20260927 22:58:47.002455 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:46.976574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.4.74.190:36669)
0927 22:58:46.977578 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:46.977592 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:46.977641 (+ 49us) server_negotiation.cc:407] Connection header received
0927 22:58:46.979848 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:46.979874 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:46.980372 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:46.980785 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:46.982932 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:46.984586 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:46.986568 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:46.987428 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:46.987690 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:46.993521 (+ 5831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:46.993531 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:46.993550 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:46.993600 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:46.997340 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:46.998210 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:46.998220 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:46.998228 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:46.998442 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:46.998886 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:46.998891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:46.998895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:46.999724 (+ 829us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:47.000099 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:47.000637 (+ 538us) server_negotiation.cc:299] Negotiation successful
0927 22:58:47.001186 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":474,"threads_started":1}
I20260927 22:58:47.005409 14819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f48f09f3fd4039bdd0784b7011b02d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36669 } }
I20260927 22:58:47.005996 14819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:47.006127 14819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70f48f09f3fd4039bdd0784b7011b02d, State: Initialized, Role: FOLLOWER
I20260927 22:58:47.006829 14819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f48f09f3fd4039bdd0784b7011b02d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36669 } }
I20260927 22:58:47.007083 14819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:58:47.007191 14819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:58:47.007344 14819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:47.010332 14819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f48f09f3fd4039bdd0784b7011b02d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36669 } }
I20260927 22:58:47.011211 14819 leader_election.cc:302] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70f48f09f3fd4039bdd0784b7011b02d; no voters:
I20260927 22:58:47.012050 14819 leader_election.cc:288] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:58:47.012233 14824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:47.013111 14824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [term 1 LEADER]: Becoming Leader. State: Replica: 70f48f09f3fd4039bdd0784b7011b02d, State: Running, Role: LEADER
I20260927 22:58:47.013793 14824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f48f09f3fd4039bdd0784b7011b02d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36669 } }
I20260927 22:58:47.014963 14819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:58:47.017053 14825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70f48f09f3fd4039bdd0784b7011b02d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f48f09f3fd4039bdd0784b7011b02d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36669 } } }
I20260927 22:58:47.017521 14825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:47.017357 14826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70f48f09f3fd4039bdd0784b7011b02d. Latest consensus state: current_term: 1 leader_uuid: "70f48f09f3fd4039bdd0784b7011b02d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f48f09f3fd4039bdd0784b7011b02d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36669 } } }
I20260927 22:58:47.018198 14826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d [sys.catalog]: This master's current role is: LEADER
I20260927 22:58:47.021116 14829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:58:47.027344 14829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:58:47.038015 14829 catalog_manager.cc:1347] Generated new cluster ID: 6ab7dc969bf14435a0d193ad2456c2f1
I20260927 22:58:47.038205 14829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:58:47.050246 14829 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:58:47.052351 14829 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:58:47.064662 14829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d: Generated new TSK 0
I20260927 22:58:47.065953 14829 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:58:47.138244 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36669
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:47.583429 14845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:47.583945 14845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:47.584126 14845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:47.592288 14845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:47.592567 14845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:47.592782 14845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:58:47.603812 14845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:47.605626 14845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:47.607313 14845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:47.626153 14853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:47.625805 14851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:47.626190 14850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:47.627061 14845 server_base.cc:1034] running on GCE node
I20260927 22:58:47.627879 14845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:47.628983 14845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:47.630240 14845 hybrid_clock.cc:648] HybridClock initialized: now 1790549927630176 us; error 60 us; skew 500 ppm
I20260927 22:58:47.630631 14845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:47 dist-test-slave-5pd3 krb5kdc[14739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549927, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:47.634178 14845 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:58:47.636278 14845 webserver.cc:460] Webserver started at http://127.4.74.129:42243/ using document root <none> and password file <none>
I20260927 22:58:47.637087 14845 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:47.637199 14845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:47.637501 14845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:47.640784 14845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "835b9e5d4b6848fab0a2cc84043f1549"
format_stamp: "Formatted at 2026-09-27 22:58:47 on dist-test-slave-5pd3"
server_key: "e586d2cf1d016b13796a4dd1943a11e2"
server_key_iv: "86f31e1045c0072d7250b9b2982aa5fe"
server_key_version: "encryptionkey@0"
I20260927 22:58:47.641537 14845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "835b9e5d4b6848fab0a2cc84043f1549"
format_stamp: "Formatted at 2026-09-27 22:58:47 on dist-test-slave-5pd3"
server_key: "e586d2cf1d016b13796a4dd1943a11e2"
server_key_iv: "86f31e1045c0072d7250b9b2982aa5fe"
server_key_version: "encryptionkey@0"
I20260927 22:58:47.647907 14845 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 22:58:47.651990 14860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:47.653759 14845 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 22:58:47.654066 14845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "835b9e5d4b6848fab0a2cc84043f1549"
format_stamp: "Formatted at 2026-09-27 22:58:47 on dist-test-slave-5pd3"
server_key: "e586d2cf1d016b13796a4dd1943a11e2"
server_key_iv: "86f31e1045c0072d7250b9b2982aa5fe"
server_key_version: "encryptionkey@0"
I20260927 22:58:47.654292 14845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:47.667253 14845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:47.671958 14845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:47.672286 14845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:47.673445 14845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:47.675038 14845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:47.675125 14845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:47.675267 14845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:47.675343 14845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:47.715615 14845 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:38805
I20260927 22:58:47.715739 14972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:38805 every 8 connection(s)
I20260927 22:58:47.717105 14845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:58:47.722149 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 14845
I20260927 22:58:47.722460 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:58:47.723451 4394 external_mini_cluster.cc:1428] Setting key cfacf8e5372b4139534067fbbe103bc8
Sep 27 22:58:47 dist-test-slave-5pd3 krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549927, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:58:47.742805 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:47.720534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:50779 (local address 127.4.74.190:36669)
0927 22:58:47.721059 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:47.721063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:47.722988 (+ 1925us) server_negotiation.cc:407] Connection header received
0927 22:58:47.724930 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:47.724937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:47.725048 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:47.725209 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:47.727267 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:47.728607 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:47.730432 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:47.731175 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:47.731343 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:47.735027 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:47.735034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:47.735037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:47.735076 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:47.738210 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:47.739241 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:47.739247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:47.739249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:47.739459 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:47.740079 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:47.740083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:47.740087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:47.740396 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:47.740531 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:47.741719 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0927 22:58:47.742201 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":284,"threads_started":1}
I20260927 22:58:47.743532 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:47.721223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36669 (local address 127.4.74.129:50779)
0927 22:58:47.722366 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:47.722439 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:47.724476 (+ 2037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:47.725521 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:47.725531 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:47.726263 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:47.727030 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:47.727050 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:47.728885 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:47.728889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:47.730228 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:47.730247 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:47.731424 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:47.731429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:47.731725 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:47.732510 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:47.732532 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:47.734749 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:47.738374 (+ 3625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:47.738387 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:47.738407 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:47.739011 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:47.739694 (+ 683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:47.739702 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:47.739708 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:47.739956 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:47.740549 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:47.740560 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:47.741042 (+ 482us) client_negotiation.cc:769] Sending connection context
0927 22:58:47.741423 (+ 381us) client_negotiation.cc:240] Negotiation successful
0927 22:58:47.742006 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":455,"threads_started":1}
I20260927 22:58:47.746181 14973 heartbeater.cc:347] Connected to a master server at 127.4.74.190:36669
I20260927 22:58:47.746711 14973 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:47.747843 14973 heartbeater.cc:511] Master 127.4.74.190:36669 requested a full tablet report, sending...
I20260927 22:58:47.750702 14784 ts_manager.cc:194] Registered new tserver with Master: 835b9e5d4b6848fab0a2cc84043f1549 (127.4.74.129:38805)
I20260927 22:58:47.752552 14784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:50779
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 22:58:47.791339 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36669
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:58:48.153472 14979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:48.153864 14979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:48.154071 14979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:48.162123 14979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:48.162308 14979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:48.162519 14979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:58:48.173419 14979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:48.174891 14979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:48.176380 14979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:48.193229 14984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:48.193152 14985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:48.193298 14987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:48.201746 14979 server_base.cc:1034] running on GCE node
I20260927 22:58:48.202602 14979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:48.203930 14979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:48.205420 14979 hybrid_clock.cc:648] HybridClock initialized: now 1790549928205316 us; error 98 us; skew 500 ppm
I20260927 22:58:48.205897 14979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:48 dist-test-slave-5pd3 krb5kdc[14739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549928, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:48.217463 14979 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:58:48.220121 14979 webserver.cc:460] Webserver started at http://127.4.74.130:42979/ using document root <none> and password file <none>
I20260927 22:58:48.220985 14979 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:48.221103 14979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:48.221503 14979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:48.224926 14979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "4da63297d2894f6ea926a340885c8972"
format_stamp: "Formatted at 2026-09-27 22:58:48 on dist-test-slave-5pd3"
server_key: "ead0d05f462281d3f721a3c02d9cbfc6"
server_key_iv: "64911943fbb2e054bb81fd112b371d25"
server_key_version: "encryptionkey@0"
I20260927 22:58:48.225750 14979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "4da63297d2894f6ea926a340885c8972"
format_stamp: "Formatted at 2026-09-27 22:58:48 on dist-test-slave-5pd3"
server_key: "ead0d05f462281d3f721a3c02d9cbfc6"
server_key_iv: "64911943fbb2e054bb81fd112b371d25"
server_key_version: "encryptionkey@0"
I20260927 22:58:48.231403 14979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260927 22:58:48.235369 14994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:48.237229 14979 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:58:48.237475 14979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "4da63297d2894f6ea926a340885c8972"
format_stamp: "Formatted at 2026-09-27 22:58:48 on dist-test-slave-5pd3"
server_key: "ead0d05f462281d3f721a3c02d9cbfc6"
server_key_iv: "64911943fbb2e054bb81fd112b371d25"
server_key_version: "encryptionkey@0"
I20260927 22:58:48.237697 14979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:48.252348 14979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:48.256870 14979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:48.257181 14979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:48.258436 14979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:48.259945 14979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:48.260021 14979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:48.260111 14979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:48.260161 14979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:48.300222 14979 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:41343
I20260927 22:58:48.300323 15106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:41343 every 8 connection(s)
I20260927 22:58:48.301642 14979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:58:48.311977 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 14979
I20260927 22:58:48.312223 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:58:48.312804 4394 external_mini_cluster.cc:1428] Setting key c0fafa756c08abf9dd0b89ea07b695ec
Sep 27 22:58:48 dist-test-slave-5pd3 krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549928, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:58:48.327360 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:48.305008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:37303 (local address 127.4.74.190:36669)
0927 22:58:48.305621 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:48.305629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:48.306827 (+ 1198us) server_negotiation.cc:407] Connection header received
0927 22:58:48.308811 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:48.308817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:48.308924 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:48.309085 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:48.310752 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:48.311921 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:48.314221 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:48.314705 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:48.314862 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:48.319711 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:48.319717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:48.319720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:48.319770 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:48.323221 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:48.324415 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:48.324421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:48.324424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:48.324635 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:48.325271 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:48.325275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:48.325277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:48.325635 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:48.325784 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:48.326772 (+ 988us) server_negotiation.cc:299] Negotiation successful
0927 22:58:48.327022 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":274,"threads_started":1}
I20260927 22:58:48.328819 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:48.305699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36669 (local address 127.4.74.130:37303)
0927 22:58:48.306617 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:48.306671 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:48.308473 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:48.309331 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:48.309340 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:48.309902 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:48.310501 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:48.310518 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:48.312140 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:48.312146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:48.314040 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:48.314053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:48.314943 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:48.314950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:48.315257 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:48.316069 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:48.316100 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:48.319311 (+ 3211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:48.323443 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:48.323457 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:48.323478 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:48.324232 (+ 754us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:48.324804 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:48.324812 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:48.324818 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:48.325089 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:48.325787 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:48.325799 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:48.326305 (+ 506us) client_negotiation.cc:769] Sending connection context
0927 22:58:48.326626 (+ 321us) client_negotiation.cc:240] Negotiation successful
0927 22:58:48.327265 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":348,"threads_started":1}
I20260927 22:58:48.330935 15107 heartbeater.cc:347] Connected to a master server at 127.4.74.190:36669
I20260927 22:58:48.331322 15107 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:48.332365 15107 heartbeater.cc:511] Master 127.4.74.190:36669 requested a full tablet report, sending...
I20260927 22:58:48.334360 14784 ts_manager.cc:194] Registered new tserver with Master: 4da63297d2894f6ea926a340885c8972 (127.4.74.130:41343)
I20260927 22:58:48.335870 14784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:37303
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:58:48.377099 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36669
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:58:48.756549 14973 heartbeater.cc:503] Master 127.4.74.190:36669 was elected leader, sending a full tablet report...
W20260927 22:58:48.781085 15113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:58:48.781507 15113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:58:48.781658 15113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:58:48.789793 15113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:58:48.790030 15113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:58:48.790228 15113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:58:48.801517 15113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:58:48.803263 15113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:58:48.804975 15113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:58:48.821218 15121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:48.821179 15119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:58:48.821836 15118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:58:48.823289 15113 server_base.cc:1034] running on GCE node
I20260927 22:58:48.824203 15113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:58:48.825503 15113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:58:48.826797 15113 hybrid_clock.cc:648] HybridClock initialized: now 1790549928826731 us; error 51 us; skew 500 ppm
I20260927 22:58:48.827247 15113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:58:48 dist-test-slave-5pd3 krb5kdc[14739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549928, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:58:48.831331 15113 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:58:48.833309 15113 webserver.cc:460] Webserver started at http://127.4.74.131:44457/ using document root <none> and password file <none>
I20260927 22:58:48.834252 15113 fs_manager.cc:359] Metadata directory not provided
I20260927 22:58:48.834368 15113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:58:48.834668 15113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 22:58:48.838291 15113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "a5d323d74aef433f82e00e0f98c8d2df"
format_stamp: "Formatted at 2026-09-27 22:58:48 on dist-test-slave-5pd3"
server_key: "ce704479e9c8d40210bc044f5551e068"
server_key_iv: "1cdd3265479daf7cd346afeba3ff3d3b"
server_key_version: "encryptionkey@0"
I20260927 22:58:48.839202 15113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "a5d323d74aef433f82e00e0f98c8d2df"
format_stamp: "Formatted at 2026-09-27 22:58:48 on dist-test-slave-5pd3"
server_key: "ce704479e9c8d40210bc044f5551e068"
server_key_iv: "1cdd3265479daf7cd346afeba3ff3d3b"
server_key_version: "encryptionkey@0"
I20260927 22:58:48.846017 15113 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260927 22:58:48.850984 15128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:48.853036 15113 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 22:58:48.853327 15113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "a5d323d74aef433f82e00e0f98c8d2df"
format_stamp: "Formatted at 2026-09-27 22:58:48 on dist-test-slave-5pd3"
server_key: "ce704479e9c8d40210bc044f5551e068"
server_key_iv: "1cdd3265479daf7cd346afeba3ff3d3b"
server_key_version: "encryptionkey@0"
I20260927 22:58:48.853603 15113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:58:48.889335 15113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:58:48.894812 15113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:58:48.895279 15113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:58:48.896749 15113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:58:48.898615 15113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:58:48.898697 15113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:48.898859 15113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:58:48.898916 15113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:58:48.940009 15113 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:44053
I20260927 22:58:48.940084 15240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:44053 every 8 connection(s)
I20260927 22:58:48.941807 15113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:58:48.947988 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 15113
I20260927 22:58:48.948210 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:58:48.948670 4394 external_mini_cluster.cc:1428] Setting key e45a6e53c3e2fe283a962e657f7bca42
Sep 27 22:58:48 dist-test-slave-5pd3 krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549928, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:58:48.969507 15243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:48.945493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:50827 (local address 127.4.74.190:36669)
0927 22:58:48.946136 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:48.946144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:48.947472 (+ 1328us) server_negotiation.cc:407] Connection header received
0927 22:58:48.950551 (+ 3079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:48.950558 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:48.950665 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:48.950804 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:48.953077 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:48.953915 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:48.955576 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:48.955989 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:48.956128 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:48.960898 (+ 4770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:48.960905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:48.960911 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:48.960966 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:48.964703 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:48.965916 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:48.965923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:48.965926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:48.966152 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:48.966706 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:48.966711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:48.966715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:48.967067 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:48.967255 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:48.968683 (+ 1428us) server_negotiation.cc:299] Negotiation successful
0927 22:58:48.969017 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":304,"threads_started":1}
I20260927 22:58:48.970842 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:48.946144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36669 (local address 127.4.74.131:50827)
0927 22:58:48.947165 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:48.947239 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:48.950019 (+ 2780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:48.951200 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:48.951212 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:48.952110 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:58:48.952893 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:48.952916 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:48.954078 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:48.954082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:48.955385 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:48.955403 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:48.956223 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:48.956233 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:48.956489 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:48.957323 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:58:48.957348 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:58:48.960604 (+ 3256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:58:48.964898 (+ 4294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:48.964912 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:48.964944 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:48.965736 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:48.966307 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:58:48.966315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:58:48.966319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:58:48.966570 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:58:48.967222 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:58:48.967232 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:58:48.967819 (+ 587us) client_negotiation.cc:769] Sending connection context
0927 22:58:48.968525 (+ 706us) client_negotiation.cc:240] Negotiation successful
0927 22:58:48.969284 (+ 759us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":375,"threads_started":1}
I20260927 22:58:48.972770 15241 heartbeater.cc:347] Connected to a master server at 127.4.74.190:36669
I20260927 22:58:48.973126 15241 heartbeater.cc:464] Registering TS with master...
I20260927 22:58:48.974076 15241 heartbeater.cc:511] Master 127.4.74.190:36669 requested a full tablet report, sending...
I20260927 22:58:48.975975 14784 ts_manager.cc:194] Registered new tserver with Master: a5d323d74aef433f82e00e0f98c8d2df (127.4.74.131:44053)
I20260927 22:58:48.976864 14784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:50827
I20260927 22:58:48.979219 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 22:58:48.999993 15243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:48.984625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44978 (local address 127.4.74.190:36669)
0927 22:58:48.984852 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:48.984857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:48.985470 (+ 613us) server_negotiation.cc:407] Connection header received
0927 22:58:48.985681 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:48.985686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:48.985757 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:48.985874 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:48.987475 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:48.988367 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:48.989901 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:48.990395 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:48.990519 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:48.993811 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:48.993816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:48.993828 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:48.993868 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:48.996802 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:48.997811 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:48.997816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:48.997856 (+ 40us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:48.998061 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:48.998485 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:48.998488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:48.998490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:48.998861 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:48.999057 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:48.999439 (+ 382us) server_negotiation.cc:299] Negotiation successful
0927 22:58:48.999725 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260927 22:58:49.004969 14784 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:58:49.012374 14784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:58:49.055357 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.029165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:44053 (local address 127.0.0.1:44368)
0927 22:58:49.044471 (+ 15306us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:49.044552 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:49.044840 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:49.046070 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:49.046077 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:49.046109 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:49.046547 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:49.046564 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.048635 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.048640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:49.051885 (+ 3245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:49.051905 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.054261 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.054268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:49.054510 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.054566 (+ 56us) client_negotiation.cc:769] Sending connection context
0927 22:58:49.054724 (+ 158us) client_negotiation.cc:240] Negotiation successful
0927 22:58:49.054941 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14870,"thread_start_us":15922,"threads_started":1}
I20260927 22:58:49.056538 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.029557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44368 (local address 127.4.74.131:44053)
0927 22:58:49.045456 (+ 15899us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:49.045465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:49.045500 (+ 35us) server_negotiation.cc:407] Connection header received
0927 22:58:49.045598 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:49.045606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:49.045858 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:49.046064 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:49.046808 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.048461 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.052088 (+ 3627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.053684 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.053813 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.054079 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:49.056011 (+ 1932us) server_negotiation.cc:299] Negotiation successful
0927 22:58:49.056191 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15716,"thread_start_us":16766,"threads_started":1}
I20260927 22:58:49.056981 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.030174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51600 (local address 127.4.74.129:38805)
0927 22:58:49.047535 (+ 17361us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:49.048263 (+ 728us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:49.048292 (+ 29us) server_negotiation.cc:407] Connection header received
0927 22:58:49.048383 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:49.048392 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:49.048642 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:49.048837 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:49.050289 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.051985 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.053891 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.055332 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.055692 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.055942 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:49.056390 (+ 448us) server_negotiation.cc:299] Negotiation successful
0927 22:58:49.056600 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17164,"thread_start_us":17002,"threads_started":1}
I20260927 22:58:49.057296 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.029401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:38805 (local address 127.0.0.1:51600)
0927 22:58:49.048018 (+ 18617us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:49.048039 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:49.048147 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:49.048929 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:49.048937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:49.048984 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:49.049999 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:49.050016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.052126 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.052131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:49.053686 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:49.053700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.056057 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.056061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:49.056190 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.056215 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 22:58:49.056330 (+ 115us) client_negotiation.cc:240] Negotiation successful
0927 22:58:49.056476 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18478,"thread_start_us":19653,"threads_started":1}
I20260927 22:58:49.058815 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.030426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.4.74.130:41343)
0927 22:58:49.046977 (+ 16551us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:49.046987 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:49.049867 (+ 2880us) server_negotiation.cc:407] Connection header received
0927 22:58:49.050216 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:49.050223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:49.050455 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:49.050613 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:49.051512 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.053029 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.055852 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.057306 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.057419 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.057652 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:49.058292 (+ 640us) server_negotiation.cc:299] Negotiation successful
0927 22:58:49.058442 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16364,"thread_start_us":16214,"threads_started":1}
I20260927 22:58:49.058844 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.029864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:41343 (local address 127.0.0.1:43312)
0927 22:58:49.049725 (+ 19861us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:49.049744 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:49.049993 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:49.050958 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:49.050961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:49.050980 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:49.051351 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:49.051359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.053215 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.053221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:49.055660 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:49.055674 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.057845 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.057850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:49.058073 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.058116 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 22:58:49.058220 (+ 104us) client_negotiation.cc:240] Negotiation successful
0927 22:58:49.058362 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19702,"thread_start_us":17864,"threads_started":1}
I20260927 22:58:49.061480 14908 tablet_service.cc:1467] Processing CreateTablet for tablet 0b3be115611b46f7bf8e9dbfc5c4db1c (DEFAULT_TABLE table=test-table [id=0918be84fab34c78bce7003a0f97dfae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:49.061480 15176 tablet_service.cc:1467] Processing CreateTablet for tablet 0b3be115611b46f7bf8e9dbfc5c4db1c (DEFAULT_TABLE table=test-table [id=0918be84fab34c78bce7003a0f97dfae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:49.062460 15042 tablet_service.cc:1467] Processing CreateTablet for tablet 0b3be115611b46f7bf8e9dbfc5c4db1c (DEFAULT_TABLE table=test-table [id=0918be84fab34c78bce7003a0f97dfae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:58:49.063231 14908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b3be115611b46f7bf8e9dbfc5c4db1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:49.063232 15176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b3be115611b46f7bf8e9dbfc5c4db1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:49.063683 15042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b3be115611b46f7bf8e9dbfc5c4db1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:58:49.076855 15260 tablet_bootstrap.cc:492] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: Bootstrap starting.
I20260927 22:58:49.080142 15260 tablet_bootstrap.cc:654] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:49.081630 15260 log.cc:824] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:49.081636 15262 tablet_bootstrap.cc:492] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: Bootstrap starting.
I20260927 22:58:49.081735 15261 tablet_bootstrap.cc:492] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: Bootstrap starting.
I20260927 22:58:49.085316 15262 tablet_bootstrap.cc:654] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:49.085894 15260 tablet_bootstrap.cc:492] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: No bootstrap required, opened a new log
I20260927 22:58:49.085925 15261 tablet_bootstrap.cc:654] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: Neither blocks nor log segments found. Creating new log.
I20260927 22:58:49.086495 15262 log.cc:824] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:49.086555 15260 ts_tablet_manager.cc:1397] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260927 22:58:49.088328 15261 log.cc:824] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: Log is configured to *not* fsync() on all Append() calls
I20260927 22:58:49.089946 15262 tablet_bootstrap.cc:492] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: No bootstrap required, opened a new log
I20260927 22:58:49.090281 15262 ts_tablet_manager.cc:1397] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260927 22:58:49.092216 15261 tablet_bootstrap.cc:492] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: No bootstrap required, opened a new log
I20260927 22:58:49.092545 15261 ts_tablet_manager.cc:1397] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260927 22:58:49.097105 15260 raft_consensus.cc:348] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 } } peers { permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 } } peers { permanent_uuid: "4da63297d2894f6ea926a340885c8972" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 41343 } }
I20260927 22:58:49.097640 15260 raft_consensus.cc:374] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:49.097747 15260 raft_consensus.cc:729] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4da63297d2894f6ea926a340885c8972, State: Initialized, Role: FOLLOWER
I20260927 22:58:49.099037 15260 consensus_queue.cc:260] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 } } peers { permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 } } peers { permanent_uuid: "4da63297d2894f6ea926a340885c8972" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 41343 } }
I20260927 22:58:49.099597 15266 raft_consensus.cc:482] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:58:49.100045 15266 raft_consensus.cc:504] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 } } peers { permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 } } peers { permanent_uuid: "4da63297d2894f6ea926a340885c8972" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 41343 } }
I20260927 22:58:49.101475 15262 raft_consensus.cc:348] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 } } peers { permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 } } peers { permanent_uuid: "4da63297d2894f6ea926a340885c8972" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 41343 } }
I20260927 22:58:49.102031 15262 raft_consensus.cc:374] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:49.102144 15262 raft_consensus.cc:729] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5d323d74aef433f82e00e0f98c8d2df, State: Initialized, Role: FOLLOWER
I20260927 22:58:49.102965 15262 consensus_queue.cc:260] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 } } peers { permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 } } peers { permanent_uuid: "4da63297d2894f6ea926a340885c8972" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 41343 } }
I20260927 22:58:49.103415 15261 raft_consensus.cc:348] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 } } peers { permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 } } peers { permanent_uuid: "4da63297d2894f6ea926a340885c8972" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 41343 } }
I20260927 22:58:49.103974 15261 raft_consensus.cc:374] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:58:49.104084 15261 raft_consensus.cc:729] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 835b9e5d4b6848fab0a2cc84043f1549, State: Initialized, Role: FOLLOWER
I20260927 22:58:49.104923 15261 consensus_queue.cc:260] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 } } peers { permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 } } peers { permanent_uuid: "4da63297d2894f6ea926a340885c8972" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 41343 } }
I20260927 22:58:49.111954 15262 ts_tablet_manager.cc:1428] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: Time spent starting tablet: real 0.022s user 0.014s sys 0.008s
I20260927 22:58:49.112468 15241 heartbeater.cc:503] Master 127.4.74.190:36669 was elected leader, sending a full tablet report...
I20260927 22:58:49.124565 15261 ts_tablet_manager.cc:1428] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: Time spent starting tablet: real 0.032s user 0.014s sys 0.016s
I20260927 22:58:49.125074 15266 leader_election.cc:288] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5d323d74aef433f82e00e0f98c8d2df (127.4.74.131:44053), 835b9e5d4b6848fab0a2cc84043f1549 (127.4.74.129:38805)
I20260927 22:58:49.125985 15107 heartbeater.cc:503] Master 127.4.74.190:36669 was elected leader, sending a full tablet report...
I20260927 22:58:49.130999 15260 ts_tablet_manager.cc:1428] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: Time spent starting tablet: real 0.044s user 0.011s sys 0.034s
I20260927 22:58:49.137180 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.106216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:42687 (local address 127.4.74.131:44053)
0927 22:58:49.107100 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:49.107110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:49.128316 (+ 21206us) server_negotiation.cc:407] Connection header received
0927 22:58:49.128969 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:49.128977 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:49.129072 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:49.129145 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:49.130073 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.131168 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.134181 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.135645 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.135845 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.135935 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:49.136682 (+ 747us) server_negotiation.cc:299] Negotiation successful
0927 22:58:49.136855 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638}
I20260927 22:58:49.137176 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.114954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:54549 (local address 127.4.74.129:38805)
0927 22:58:49.115236 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:49.115246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:49.128670 (+ 13424us) server_negotiation.cc:407] Connection header received
0927 22:58:49.129487 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:49.129494 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:49.129581 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:49.129645 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:58:49.130543 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.131572 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.134178 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.135641 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.135845 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.135935 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:49.136682 (+ 747us) server_negotiation.cc:299] Negotiation successful
0927 22:58:49.136878 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 22:58:49.137326 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.104253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:44053 (local address 127.4.74.130:42687)
0927 22:58:49.128187 (+ 23934us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:49.128210 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:49.128797 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:49.129403 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:49.129410 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:49.129439 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:49.129902 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:49.129912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.131358 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.131362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:49.133895 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:49.133914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.136012 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.136020 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:49.136289 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.136430 (+ 141us) client_negotiation.cc:769] Sending connection context
0927 22:58:49.136550 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 22:58:49.136764 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23748,"mutex_wait_us":168,"thread_start_us":21503,"threads_started":1}
I20260927 22:58:49.137382 15271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.114280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:38805 (local address 127.4.74.130:54549)
0927 22:58:49.128566 (+ 14286us) negotiation.cc:107] Waiting for socket to connect
0927 22:58:49.128584 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 22:58:49.129317 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:58:49.129897 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:58:49.129901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:58:49.129923 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:58:49.130356 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:49.130367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.131747 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.131753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:49.133992 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:58:49.134005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.136012 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.136020 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:58:49.136308 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.136431 (+ 123us) client_negotiation.cc:769] Sending connection context
0927 22:58:49.136549 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 22:58:49.136764 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14170,"mutex_wait_us":191,"thread_start_us":13504,"threads_started":1}
I20260927 22:58:49.139009 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b3be115611b46f7bf8e9dbfc5c4db1c" candidate_uuid: "4da63297d2894f6ea926a340885c8972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5d323d74aef433f82e00e0f98c8d2df" is_pre_election: true
I20260927 22:58:49.139528 14928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b3be115611b46f7bf8e9dbfc5c4db1c" candidate_uuid: "4da63297d2894f6ea926a340885c8972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "835b9e5d4b6848fab0a2cc84043f1549" is_pre_election: true
I20260927 22:58:49.139732 15196 raft_consensus.cc:2454] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4da63297d2894f6ea926a340885c8972 in term 0.
I20260927 22:58:49.140072 14928 raft_consensus.cc:2454] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4da63297d2894f6ea926a340885c8972 in term 0.
I20260927 22:58:49.140628 14995 leader_election.cc:302] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4da63297d2894f6ea926a340885c8972, a5d323d74aef433f82e00e0f98c8d2df; no voters:
I20260927 22:58:49.141163 15267 raft_consensus.cc:2781] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:58:49.141358 15267 raft_consensus.cc:482] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:58:49.141535 15267 raft_consensus.cc:3037] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:49.144152 15267 raft_consensus.cc:504] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 } } peers { permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 } } peers { permanent_uuid: "4da63297d2894f6ea926a340885c8972" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 41343 } }
I20260927 22:58:49.144865 15267 leader_election.cc:288] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [CANDIDATE]: Term 1 election: Requested vote from peers a5d323d74aef433f82e00e0f98c8d2df (127.4.74.131:44053), 835b9e5d4b6848fab0a2cc84043f1549 (127.4.74.129:38805)
I20260927 22:58:49.145408 14928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b3be115611b46f7bf8e9dbfc5c4db1c" candidate_uuid: "4da63297d2894f6ea926a340885c8972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "835b9e5d4b6848fab0a2cc84043f1549"
I20260927 22:58:49.145474 15196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b3be115611b46f7bf8e9dbfc5c4db1c" candidate_uuid: "4da63297d2894f6ea926a340885c8972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5d323d74aef433f82e00e0f98c8d2df"
I20260927 22:58:49.145627 14928 raft_consensus.cc:3037] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:49.145793 15196 raft_consensus.cc:3037] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:58:49.148063 14928 raft_consensus.cc:2454] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4da63297d2894f6ea926a340885c8972 in term 1.
I20260927 22:58:49.148614 14997 leader_election.cc:302] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4da63297d2894f6ea926a340885c8972, 835b9e5d4b6848fab0a2cc84043f1549; no voters:
I20260927 22:58:49.148685 15196 raft_consensus.cc:2454] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4da63297d2894f6ea926a340885c8972 in term 1.
I20260927 22:58:49.149070 15267 raft_consensus.cc:2781] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:58:49.149336 15267 raft_consensus.cc:686] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 1 LEADER]: Becoming Leader. State: Replica: 4da63297d2894f6ea926a340885c8972, State: Running, Role: LEADER
I20260927 22:58:49.149868 15267 consensus_queue.cc:237] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 } } peers { permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 } } peers { permanent_uuid: "4da63297d2894f6ea926a340885c8972" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 41343 } }
I20260927 22:58:49.155494 14784 catalog_manager.cc:5495] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4da63297d2894f6ea926a340885c8972 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "4da63297d2894f6ea926a340885c8972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4da63297d2894f6ea926a340885c8972" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 41343 } health_report { overall_health: HEALTHY } } }
W20260927 22:58:49.194161 15242 tablet.cc:2369] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:58:49.199512 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.192859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43328 (local address 127.4.74.130:41343)
0927 22:58:49.193164 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:49.193173 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:49.193294 (+ 121us) server_negotiation.cc:407] Connection header received
0927 22:58:49.193602 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:49.193607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:49.193717 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:49.193917 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:58:49.194327 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.195191 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.196732 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.197194 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.197348 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.197954 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:58:49.198796 (+ 842us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:58:49.198972 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:49.199083 (+ 111us) server_negotiation.cc:299] Negotiation successful
0927 22:58:49.199186 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260927 22:58:49.211184 15196 raft_consensus.cc:1261] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df [term 1 FOLLOWER]: Refusing update from remote peer 4da63297d2894f6ea926a340885c8972: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:49.211184 14928 raft_consensus.cc:1261] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549 [term 1 FOLLOWER]: Refusing update from remote peer 4da63297d2894f6ea926a340885c8972: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:58:49.212529 15267 consensus_queue.cc:1035] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5d323d74aef433f82e00e0f98c8d2df" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:58:49.212929 15266 consensus_queue.cc:1035] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "835b9e5d4b6848fab0a2cc84043f1549" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 38805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 22:58:49.219980 14974 tablet.cc:2369] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:58:49.245229 14783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:49.245605 14783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:58:49.250382 14783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70f48f09f3fd4039bdd0784b7011b02d: Sending DeleteTablet for 3 replicas of tablet 0b3be115611b46f7bf8e9dbfc5c4db1c
I20260927 22:58:49.251977 15176 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3be115611b46f7bf8e9dbfc5c4db1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:49 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:44368
I20260927 22:58:49.251984 15042 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3be115611b46f7bf8e9dbfc5c4db1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:49 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:43312
I20260927 22:58:49.252044 14908 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b3be115611b46f7bf8e9dbfc5c4db1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:58:49 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:51600
I20260927 22:58:49.255772 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 14845
I20260927 22:58:49.268903 15286 tablet_replica.cc:330] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: stopping tablet replica
I20260927 22:58:49.270072 15284 tablet_replica.cc:330] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: stopping tablet replica
I20260927 22:58:49.270360 15286 raft_consensus.cc:2229] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:49.270783 15285 tablet_replica.cc:330] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: stopping tablet replica
I20260927 22:58:49.271206 15286 raft_consensus.cc:2258] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:49.271723 15284 raft_consensus.cc:2229] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:58:49.271939 15285 raft_consensus.cc:2229] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:58:49.272692 15285 raft_consensus.cc:2258] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:49.273270 15284 raft_consensus.cc:2258] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:58:49.274709 15286 ts_tablet_manager.cc:1905] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:49.275887 15284 ts_tablet_manager.cc:1905] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:49.277391 15285 ts_tablet_manager.cc:1905] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:58:49.286087 15286 ts_tablet_manager.cc:1918] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:49.286317 15286 log.cc:1196] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/0b3be115611b46f7bf8e9dbfc5c4db1c
I20260927 22:58:49.286190 15284 ts_tablet_manager.cc:1918] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:49.286672 15284 log.cc:1196] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/0b3be115611b46f7bf8e9dbfc5c4db1c
I20260927 22:58:49.286672 15285 ts_tablet_manager.cc:1918] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:58:49.286895 15285 log.cc:1196] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/0b3be115611b46f7bf8e9dbfc5c4db1c
I20260927 22:58:49.286911 15286 ts_tablet_manager.cc:1939] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 835b9e5d4b6848fab0a2cc84043f1549: Deleting consensus metadata
Sep 27 22:58:49 dist-test-slave-5pd3 krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:58:49.287528 15284 ts_tablet_manager.cc:1939] T 0b3be115611b46f7bf8e9dbfc5c4db1c P a5d323d74aef433f82e00e0f98c8d2df: Deleting consensus metadata
I20260927 22:58:49.287606 15285 ts_tablet_manager.cc:1939] T 0b3be115611b46f7bf8e9dbfc5c4db1c P 4da63297d2894f6ea926a340885c8972: Deleting consensus metadata
I20260927 22:58:49.289086 14770 catalog_manager.cc:4841] TS 835b9e5d4b6848fab0a2cc84043f1549 (127.4.74.129:38805): tablet 0b3be115611b46f7bf8e9dbfc5c4db1c (table test-table [id=0918be84fab34c78bce7003a0f97dfae]) successfully deleted
I20260927 22:58:49.289592 14768 catalog_manager.cc:4841] TS a5d323d74aef433f82e00e0f98c8d2df (127.4.74.131:44053): tablet 0b3be115611b46f7bf8e9dbfc5c4db1c (table test-table [id=0918be84fab34c78bce7003a0f97dfae]) successfully deleted
I20260927 22:58:49.290314 14771 catalog_manager.cc:4841] TS 4da63297d2894f6ea926a340885c8972 (127.4.74.130:41343): tablet 0b3be115611b46f7bf8e9dbfc5c4db1c (table test-table [id=0918be84fab34c78bce7003a0f97dfae]) successfully deleted
I20260927 22:58:49.296041 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.257323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51616 (local address 127.4.74.129:38805)
0927 22:58:49.257578 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:49.257585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:49.269176 (+ 11591us) server_negotiation.cc:407] Connection header received
0927 22:58:49.269589 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:49.269598 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:49.269757 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:49.269905 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:49.272071 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.273265 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.275249 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.275963 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.276080 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.288887 (+ 12807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:49.288909 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:49.288932 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:49.288999 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:49.292691 (+ 3692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:49.293609 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:49.293618 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:49.293620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:49.293776 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:49.294196 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:49.294205 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:49.294207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:49.294630 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:49.294989 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:49.295297 (+ 308us) server_negotiation.cc:299] Negotiation successful
0927 22:58:49.295635 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260927 22:58:49.296885 14968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:49.413160 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 14845
I20260927 22:58:49.434458 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 14979
Sep 27 22:58:49 dist-test-slave-5pd3 krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:58:49.458346 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.435741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43332 (local address 127.4.74.130:41343)
0927 22:58:49.436017 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:49.436024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:49.436054 (+ 30us) server_negotiation.cc:407] Connection header received
0927 22:58:49.436132 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:49.436140 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:49.436249 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:49.436386 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:49.437948 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.439259 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.440830 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.441579 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.441731 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.448130 (+ 6399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:49.448154 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:49.448180 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:49.448502 (+ 322us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:49.453443 (+ 4941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:49.454609 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:49.454624 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:49.454627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:49.454818 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:49.455348 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:49.455353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:49.455356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:49.456053 (+ 697us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:49.456516 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:49.457258 (+ 742us) server_negotiation.cc:299] Negotiation successful
0927 22:58:49.457916 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260927 22:58:49.460727 15102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:49.589987 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 14979
I20260927 22:58:49.612183 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 15113
Sep 27 22:58:49 dist-test-slave-5pd3 krb5kdc[14739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:58:49.630553 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:58:49.613204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44378 (local address 127.4.74.131:44053)
0927 22:58:49.613407 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 22:58:49.613412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:58:49.613511 (+ 99us) server_negotiation.cc:407] Connection header received
0927 22:58:49.613704 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:58:49.613708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:58:49.613777 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:58:49.613900 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:58:49.615876 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.616766 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.618321 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:58:49.618743 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:58:49.618867 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:58:49.624399 (+ 5532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:58:49.624416 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:58:49.624432 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:58:49.624473 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:58:49.627439 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:49.628444 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:49.628452 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:49.628454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:49.628609 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:58:49.629036 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:58:49.629041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:58:49.629043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:58:49.629475 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:58:49.629768 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0927 22:58:49.630073 (+ 305us) server_negotiation.cc:299] Negotiation successful
0927 22:58:49.630254 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 22:58:49.631393 15236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:49.726560 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 15113
I20260927 22:58:49.746819 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 14752
I20260927 22:58:49.748332 14814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:58:49.864902 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 14752
2026-09-27T22:58:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4778 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:58:49 dist-test-slave-5pd3 krb5kdc[15294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:58:49 dist-test-slave-5pd3 krb5kdc[15294](info): set up 2 sockets
Sep 27 22:58:49 dist-test-slave-5pd3 krb5kdc[15294](info): commencing operation
krb5kdc: starting...
W20260927 22:58:51.000836 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:58:51 dist-test-slave-5pd3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:58:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:58:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/rangeradmin_127.4.74.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/rangeradmin_127.4.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/rangerlookup_127.4.74.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/rangerlookup_127.4.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/HTTP_127.4.74.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/HTTP_127.4.74.148@KRBTEST.COM.keytab.
I20260927 22:58:51.408419 4394 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskEVsig8/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/postgres -l logfile start
2026-09-27 22:58:54.539 UTC [15324] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 22:58:54.539 UTC [15324] LOG: listening on IPv4 address "127.4.74.148", port 49065
2026-09-27 22:58:54.544 UTC [15324] LOG: listening on Unix socket "/tmp/.s.PGSQL.49065"
2026-09-27 22:58:54.574 UTC [15327] LOG: database system was shut down at 2026-09-27 22:58:52 UTC
2026-09-27 22:58:54.578 UTC [15324] LOG: database system is ready to accept connections
I20260927 22:58:55.596261 4394 mini_postgres.cc:96] Postgres bound to 49065
2026-09-27 22:58:55.617 UTC [15335] FATAL: database "slave" does not exist
127.4.74.148:49065 - accepting connections
I20260927 22:58:55.618376 4394 mini_ranger.cc:161] Starting Ranger...
I20260927 22:58:55.651129 4394 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 22:58:56.311101 4394 mini_ranger.cc:87] Created ranger Postgres database
I20260927 22:58:56.311290 4394 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-admin
2026-09-27 22:58:56,639 [I] DB FLAVOR :POSTGRES
2026-09-27 22:58:56,640 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 22:58:56,640 [I] Checking connection..
2026-09-27 22:58:56,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 22:58:56,931 [I] Checking connection passed.
2026-09-27 22:58:56,932 [I] --------- Verifying version history table ---------
2026-09-27 22:58:56,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 22:58:57,245 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 22:58:57,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 22:58:57,507 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 22:58:57,508 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 22:58:57,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 22:58:57.763 UTC [15399] WARNING: there is no transaction in progress
2026-09-27 22:58:57,770 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 22:58:57,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 22:58:58,052 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 22:58:58,052 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 22:58:58,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 22:58:58,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 22:58:58,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5pd3.c.gcp-upstream.internal','N') ;"
2026-09-27 22:58:58,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 22:58:59,194 [I] Table x_portal_user does not exist in database ranger
2026-09-27 22:58:59,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 22:58:59,466 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 22:58:59,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 22:58:59,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 22:59:00,044 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 22:59:00,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 22:59:01.208 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.228 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.248 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.267 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.287 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.308 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.380 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.388 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.407 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.421 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.434 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.450 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.461 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.468 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.476 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:01.484 UTC [15507] WARNING: there is no transaction in progress
2026-09-27 22:59:02,412 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 22:59:02,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 22:59:02,668 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 22:59:02,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 22:59:02,929 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 22:59:02,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 22:59:03,183 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 22:59:03,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 22:59:03,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 22:59:03,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5pd3.c.gcp-upstream.internal';"
2026-09-27 22:59:04,008 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 22:59:04,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 22:59:04,316 [I] DB_PATCHES have already been applied
I20260927 22:59:04.322322 4394 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-admin:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskEVsig8/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 22:59:04.322425 4394 mini_ranger.cc:193] Using host: 127.4.74.148
I20260927 22:59:04.341262 4394 mini_ranger.cc:227] Ranger admin URL: http://127.4.74.148:43907
Sep 27, 2026 10:59:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 10:59:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 10:59:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 10:59:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 10:59:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 10:59:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 10:59:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 22:59:05.535319 4394 mini_ranger.cc:160] Time spent starting Ranger: real 9.917s user 0.007s sys 0.046s
Sep 27, 2026 10:59:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 10:59:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 27, 2026 10:59:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 10:59:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 10:59:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 22:59:30.194455 4394 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/rangerkms_127.4.74.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/rangerkms_127.4.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.74.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.74.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.74.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/HTTP_127.4.74.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.74.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/HTTP_127.4.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 22:59:30 dist-test-slave-5pd3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549970, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 22:59:30.412643 4394 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 22:59:30.439692 4394 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 22:59:31.541234 4394 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 22:59:31.541445 4394 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-kms
I20260927 22:59:31.541568 4394 mini_ranger_kms.cc:227] Using postgres at 127.4.74.148:49065
2026-09-27 22:59:31,978 [I] DB FLAVOR :POSTGRES
2026-09-27 22:59:31,978 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 22:59:31,978 [I] Checking connection
2026-09-27 22:59:31,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 22:59:32,260 [I] connection success
2026-09-27 22:59:32,261 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 22:59:32,261 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 22:59:32,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 22:59:32,541 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 22:59:32,542 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 22:59:32,542 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 22:59:32,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:49065/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 22:59:32,825 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 22:59:33.388531 4394 mini_ranger_kms.cc:310] Created kms service
I20260927 22:59:33.623955 4394 mini_ranger_kms.cc:326] Created kudu user
I20260927 22:59:33.677313 4394 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 22:59:33 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549970, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 22:59:34.177433 4394 mini_ranger_kms.cc:384] Added ranger policy
I20260927 22:59:34.177815 4394 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-kms:/tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 22:59:34.177884 4394 mini_ranger_kms.cc:240] Using host: 127.4.74.148
I20260927 22:59:34.204910 4394 mini_ranger_kms.cc:276] Ranger KMS PID: 15688
I20260927 22:59:34.205188 4394 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.74.148:54169
Sep 27, 2026 10:59:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 10:59:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 10:59:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 10:59:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 10:59:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 10:59:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 22:59:36.522221 4394 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.110s user 0.126s sys 0.334s
I20260927 22:59:36.522491 4394 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 22:59:36.522718 4394 mini_ranger_kms.cc:401] 127.4.74.148:54169/kms/v1/keys
Sep 27 22:59:36 dist-test-slave-5pd3 krb5kdc[15294](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790549976, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.74.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 22:59:37 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549976, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.74.148@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
Sep 27 22:59:37 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549976, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.74.148@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
Sep 27 22:59:40 dist-test-slave-5pd3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 22:59:40.177981 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:33763
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:35037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:33763
--ranger_config_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:54169/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:59:40.615535 15738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:59:40.616046 15738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:59:40.616222 15738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:59:40.624562 15738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 22:59:40.624748 15738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:59:40.624866 15738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:59:40.624970 15738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 22:59:40.625073 15738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 22:59:40.636595 15738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.74.148:54169/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:33763
--ranger_config_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:33763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:59:40.638039 15738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:59:40.639451 15738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:59:40.653566 15743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:59:40.653671 15746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:59:40.653949 15744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:59:40.656348 15738 server_base.cc:1034] running on GCE node
I20260927 22:59:40.657733 15738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:59:40.660444 15738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:59:40.661768 15738 hybrid_clock.cc:648] HybridClock initialized: now 1790549980661694 us; error 83 us; skew 500 ppm
I20260927 22:59:40.662393 15738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:59:40 dist-test-slave-5pd3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549980, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:59:40.667508 15738 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 22:59:40.669574 15738 webserver.cc:460] Webserver started at http://127.4.74.190:38773/ using document root <none> and password file <none>
I20260927 22:59:40.670396 15738 fs_manager.cc:359] Metadata directory not provided
I20260927 22:59:40.670538 15738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:59:40.670864 15738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 22:59:40 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549980, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 22:59:40.743870 15738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "0da76da7e9a143fa962fc02b5c7a3b2f"
format_stamp: "Formatted at 2026-09-27 22:59:40 on dist-test-slave-5pd3"
server_key: "8e7637f4d54d43f17985be45a57b2274"
server_key_iv: "1530dc512171bd84eb39593f778a070e"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:40.745110 15738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "0da76da7e9a143fa962fc02b5c7a3b2f"
format_stamp: "Formatted at 2026-09-27 22:59:40 on dist-test-slave-5pd3"
server_key: "8e7637f4d54d43f17985be45a57b2274"
server_key_iv: "1530dc512171bd84eb39593f778a070e"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:40.753597 15738 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260927 22:59:40.776386 15753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:59:40.778903 15738 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260927 22:59:40.779199 15738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "0da76da7e9a143fa962fc02b5c7a3b2f"
format_stamp: "Formatted at 2026-09-27 22:59:40 on dist-test-slave-5pd3"
server_key: "8e7637f4d54d43f17985be45a57b2274"
server_key_iv: "1530dc512171bd84eb39593f778a070e"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:40.779526 15738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:59:40.818007 15738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:59:40.823868 15738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:59:40.824262 15738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:59:40.847942 15738 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:33763
I20260927 22:59:40.847942 15804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:33763 every 8 connection(s)
I20260927 22:59:40.850047 15738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 22:59:40.854935 15805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:59:40.860548 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 15738
I20260927 22:59:40.860823 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 22:59:40.861693 4394 external_mini_cluster.cc:1428] Setting key a45c1ddeff6769db53af946f8f51085e
I20260927 22:59:40.870332 15805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f: Bootstrap starting.
I20260927 22:59:40.875392 15805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f: Neither blocks nor log segments found. Creating new log.
I20260927 22:59:40.876966 15805 log.cc:824] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f: Log is configured to *not* fsync() on all Append() calls
Sep 27 22:59:40 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:59:40.880919 15805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f: No bootstrap required, opened a new log
I20260927 22:59:40.890722 15808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:40.864812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55182 (local address 127.4.74.190:33763)
0927 22:59:40.866400 (+ 1588us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:40.866414 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:40.866471 (+ 57us) server_negotiation.cc:407] Connection header received
0927 22:59:40.868457 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:40.868486 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:40.868966 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:40.869464 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:59:40.870999 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:40.872481 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:40.873757 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:40.874468 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:40.874668 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:40.879497 (+ 4829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:59:40.879509 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:59:40.879535 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:59:40.879605 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:59:40.884220 (+ 4615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:40.885374 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:40.885385 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:40.885398 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:40.885727 (+ 329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:40.886213 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:40.886219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:40.886247 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:40.887287 (+ 1040us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:59:40.887680 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:40.888254 (+ 574us) server_negotiation.cc:299] Negotiation successful
0927 22:59:40.888852 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1280,"thread_start_us":971,"threads_started":1}
I20260927 22:59:40.891932 15805 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da76da7e9a143fa962fc02b5c7a3b2f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 33763 } }
I20260927 22:59:40.892397 15805 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:59:40.892530 15805 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da76da7e9a143fa962fc02b5c7a3b2f, State: Initialized, Role: FOLLOWER
I20260927 22:59:40.893285 15805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da76da7e9a143fa962fc02b5c7a3b2f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 33763 } }
I20260927 22:59:40.893559 15805 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 22:59:40.893730 15805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 22:59:40.893936 15805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:59:40.896710 15805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da76da7e9a143fa962fc02b5c7a3b2f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 33763 } }
I20260927 22:59:40.897316 15805 leader_election.cc:302] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0da76da7e9a143fa962fc02b5c7a3b2f; no voters:
I20260927 22:59:40.898021 15805 leader_election.cc:288] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 22:59:40.898272 15810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:59:40.899575 15810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [term 1 LEADER]: Becoming Leader. State: Replica: 0da76da7e9a143fa962fc02b5c7a3b2f, State: Running, Role: LEADER
I20260927 22:59:40.900475 15805 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 22:59:40.900470 15810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da76da7e9a143fa962fc02b5c7a3b2f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 33763 } }
I20260927 22:59:40.910948 15811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0da76da7e9a143fa962fc02b5c7a3b2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da76da7e9a143fa962fc02b5c7a3b2f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 33763 } } }
I20260927 22:59:40.911595 15811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [sys.catalog]: This master's current role is: LEADER
I20260927 22:59:40.912366 15812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0da76da7e9a143fa962fc02b5c7a3b2f. Latest consensus state: current_term: 1 leader_uuid: "0da76da7e9a143fa962fc02b5c7a3b2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da76da7e9a143fa962fc02b5c7a3b2f" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 33763 } } }
I20260927 22:59:40.912681 15812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f [sys.catalog]: This master's current role is: LEADER
I20260927 22:59:40.934537 15817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 22:59:40.941335 15817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 22:59:40.942946 15805 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 22:59:40.964520 15817 catalog_manager.cc:1347] Generated new cluster ID: 31054766a89541978894313059aa5f98
I20260927 22:59:40.964831 15817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 22:59:40.996928 15817 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 22:59:40.999403 15817 catalog_manager.cc:1504] Loading token signing keys...
I20260927 22:59:41.009970 15817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f: Generated new TSK 0
I20260927 22:59:41.011606 15817 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 22:59:41.909029 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:54169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:33763
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:35037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 22:59:42 dist-test-slave-5pd3 krb5kdc[15294](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790549982, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 22:59:42.434415 15851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:59:42.434775 15851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:59:42.434947 15851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:59:42.442873 15851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:59:42.443034 15851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:59:42.443228 15851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 22:59:42.454185 15851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.74.148:54169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:33763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:59:42.455737 15851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:59:42.457619 15851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:59:42.491611 15861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:59:42.493846 15862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:59:42.501510 15864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:59:42.502038 15851 server_base.cc:1034] running on GCE node
I20260927 22:59:42.503607 15851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:59:42.505368 15851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:59:42.506719 15851 hybrid_clock.cc:648] HybridClock initialized: now 1790549982506634 us; error 96 us; skew 500 ppm
I20260927 22:59:42.507217 15851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:59:42 dist-test-slave-5pd3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549982, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:59:42.512271 15851 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 22:59:42.514690 15851 webserver.cc:460] Webserver started at http://127.4.74.129:42533/ using document root <none> and password file <none>
I20260927 22:59:42.515779 15851 fs_manager.cc:359] Metadata directory not provided
I20260927 22:59:42.515955 15851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:59:42.516397 15851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 22:59:42 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549982, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 22:59:42.550032 15851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "376654ad9a1a4d51b4d24bc19a618491"
format_stamp: "Formatted at 2026-09-27 22:59:42 on dist-test-slave-5pd3"
server_key: "19d085af729a45527486dac1e256a8d3"
server_key_iv: "3bcd6a3667d87a6a4e699f5088732f0e"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:42.551419 15851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "376654ad9a1a4d51b4d24bc19a618491"
format_stamp: "Formatted at 2026-09-27 22:59:42 on dist-test-slave-5pd3"
server_key: "19d085af729a45527486dac1e256a8d3"
server_key_iv: "3bcd6a3667d87a6a4e699f5088732f0e"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:42.559722 15851 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260927 22:59:42.592623 15871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:59:42.595531 15851 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.002s sys 0.009s
I20260927 22:59:42.595959 15851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "376654ad9a1a4d51b4d24bc19a618491"
format_stamp: "Formatted at 2026-09-27 22:59:42 on dist-test-slave-5pd3"
server_key: "19d085af729a45527486dac1e256a8d3"
server_key_iv: "3bcd6a3667d87a6a4e699f5088732f0e"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:42.596606 15851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:59:42.650473 15851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:59:42.657399 15851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:59:42.657904 15851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:59:42.665868 15851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:59:42.675014 15851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:59:42.675280 15851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260927 22:59:42.675568 15851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:59:42.675748 15851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 22:59:42 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549982, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
Sep 27 22:59:43 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549982, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 22:59:43.333185 15984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:44789 every 8 connection(s)
I20260927 22:59:43.333710 15851 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:44789
I20260927 22:59:43.336236 15851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 22:59:43.349035 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 15851
I20260927 22:59:43.349345 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 22:59:43.350775 4394 external_mini_cluster.cc:1428] Setting key 33faaf8558b06f785eacf0ebc87c82f9
Sep 27 22:59:43 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549982, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
I20260927 22:59:43.413417 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:43.343711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:38287 (local address 127.4.74.190:33763)
0927 22:59:43.363242 (+ 19531us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:43.363249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:43.363271 (+ 22us) server_negotiation.cc:407] Connection header received
0927 22:59:43.363348 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:43.363355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:43.363457 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:43.363609 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:59:43.367384 (+ 3775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:43.368920 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:43.372546 (+ 3626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:43.373378 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:43.373510 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:43.389904 (+ 16394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:59:43.389917 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:59:43.389927 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:59:43.390005 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:59:43.394369 (+ 4364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:43.401493 (+ 7124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:43.401502 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:43.401505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:43.401881 (+ 376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:43.407436 (+ 5555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:43.407462 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:43.407469 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:43.408094 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:59:43.408279 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:43.412440 (+ 4161us) server_negotiation.cc:299] Negotiation successful
0927 22:59:43.412858 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19361,"thread_start_us":21371,"threads_started":1}
I20260927 22:59:43.416088 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:43.344503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:33763 (local address 127.4.74.129:38287)
0927 22:59:43.358871 (+ 14368us) negotiation.cc:107] Waiting for socket to connect
0927 22:59:43.358943 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0927 22:59:43.361823 (+ 2880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:59:43.364515 (+ 2692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:59:43.364532 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:59:43.365417 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:59:43.367021 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:43.367085 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:43.369246 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:43.369253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:43.372274 (+ 3021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:43.372303 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:43.374008 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:43.374020 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:43.374416 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:43.375435 (+ 1019us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:59:43.375466 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:59:43.384591 (+ 9125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:59:43.400107 (+ 15516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:59:43.400123 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:59:43.400150 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:59:43.401260 (+ 1110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:59:43.402090 (+ 830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:59:43.402094 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:59:43.402098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:59:43.405140 (+ 3042us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:59:43.408488 (+ 3348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:59:43.408504 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:59:43.409235 (+ 731us) client_negotiation.cc:769] Sending connection context
0927 22:59:43.413379 (+ 4144us) client_negotiation.cc:240] Negotiation successful
0927 22:59:43.414209 (+ 830us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13911,"thread_start_us":17772,"threads_started":1}
I20260927 22:59:43.418710 15985 heartbeater.cc:347] Connected to a master server at 127.4.74.190:33763
I20260927 22:59:43.419137 15985 heartbeater.cc:464] Registering TS with master...
I20260927 22:59:43.420372 15985 heartbeater.cc:511] Master 127.4.74.190:33763 requested a full tablet report, sending...
I20260927 22:59:43.425519 15770 ts_manager.cc:194] Registered new tserver with Master: 376654ad9a1a4d51b4d24bc19a618491 (127.4.74.129:44789)
I20260927 22:59:43.429209 15770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:38287
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
W20260927 22:59:43.464921 4394 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.4.74.130: real 0.109s user 0.005s sys 0.015s
I20260927 22:59:43.466099 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:54169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:33763
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:35037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 22:59:43.884444 15992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:59:43.884743 15992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:59:43.884835 15992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:59:43.892602 15992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:59:43.892748 15992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:59:43.892865 15992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 22:59:43.903254 15992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.74.148:54169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:33763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:59:43.904680 15992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:59:43.906045 15992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:59:43.921175 16005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:59:43.921233 16007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:59:43.921885 16004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:59:43.922283 15992 server_base.cc:1034] running on GCE node
I20260927 22:59:43.923086 15992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:59:43.924494 15992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:59:43.925807 15992 hybrid_clock.cc:648] HybridClock initialized: now 1790549983925730 us; error 75 us; skew 500 ppm
I20260927 22:59:43.926265 15992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:59:43 dist-test-slave-5pd3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549983, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:59:43.930562 15992 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 22:59:43.932673 15992 webserver.cc:460] Webserver started at http://127.4.74.130:36099/ using document root <none> and password file <none>
I20260927 22:59:43.933620 15992 fs_manager.cc:359] Metadata directory not provided
I20260927 22:59:43.933791 15992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:59:43.934233 15992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 22:59:43 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549983, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 22:59:43.952776 15992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "69fdcfebccdb4dc4808c98771f26289a"
format_stamp: "Formatted at 2026-09-27 22:59:43 on dist-test-slave-5pd3"
server_key: "1031f847576286c08aaac1f85dc2399d"
server_key_iv: "d702dba6dd0dc936db1cfeba23813e27"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:43.953663 15992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "69fdcfebccdb4dc4808c98771f26289a"
format_stamp: "Formatted at 2026-09-27 22:59:43 on dist-test-slave-5pd3"
server_key: "1031f847576286c08aaac1f85dc2399d"
server_key_iv: "d702dba6dd0dc936db1cfeba23813e27"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:43.961196 15992 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260927 22:59:43.976514 16014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:59:43.978644 15992 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260927 22:59:43.978885 15992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "69fdcfebccdb4dc4808c98771f26289a"
format_stamp: "Formatted at 2026-09-27 22:59:43 on dist-test-slave-5pd3"
server_key: "1031f847576286c08aaac1f85dc2399d"
server_key_iv: "d702dba6dd0dc936db1cfeba23813e27"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:43.979140 15992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:59:43.993817 15992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:59:43.999022 15992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:59:43.999424 15992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:59:44.002189 15992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:59:44.004071 15992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:59:44.004179 15992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:59:44.004334 15992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:59:44.004398 15992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:59:44.058341 15992 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:42305
I20260927 22:59:44.058449 16126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:42305 every 8 connection(s)
I20260927 22:59:44.059808 15992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 22:59:44.069039 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 15992
I20260927 22:59:44.069276 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 22:59:44.069950 4394 external_mini_cluster.cc:1428] Setting key 3a1bd26d7d48aceaa080ebd277e813b7
Sep 27 22:59:44 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549983, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:59:44.086702 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.064048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:33763 (local address 127.4.74.130:53625)
0927 22:59:44.064901 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0927 22:59:44.064957 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 22:59:44.066643 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:59:44.067730 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:59:44.067739 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:59:44.068167 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:59:44.068778 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.068796 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.070073 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.070081 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.071669 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.071684 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.072937 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.072944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.073240 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.074198 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:59:44.074234 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:59:44.077775 (+ 3541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:59:44.081995 (+ 4220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:59:44.082007 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:59:44.082035 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:59:44.082680 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:59:44.083124 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:59:44.083129 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:59:44.083133 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:59:44.083360 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:59:44.084230 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:59:44.084239 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:59:44.084622 (+ 383us) client_negotiation.cc:769] Sending connection context
0927 22:59:44.085008 (+ 386us) client_negotiation.cc:240] Negotiation successful
0927 22:59:44.085494 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":374,"threads_started":1}
I20260927 22:59:44.087841 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.063609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:53625 (local address 127.4.74.190:33763)
0927 22:59:44.067176 (+ 3567us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:44.067181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:44.067200 (+ 19us) server_negotiation.cc:407] Connection header received
0927 22:59:44.067257 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:44.067261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:44.067329 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:44.067459 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:59:44.068965 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.069917 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.071848 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.072644 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.072836 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.078165 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:59:44.078176 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:59:44.078196 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:59:44.078255 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:59:44.081796 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:44.082818 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:44.082827 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:44.082830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:44.083014 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:44.083698 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:44.083702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:44.083705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:44.084079 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:59:44.084262 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:44.087251 (+ 2989us) server_negotiation.cc:299] Negotiation successful
0927 22:59:44.087500 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":3410,"thread_start_us":3265,"threads_started":1}
I20260927 22:59:44.089150 16127 heartbeater.cc:347] Connected to a master server at 127.4.74.190:33763
I20260927 22:59:44.089471 16127 heartbeater.cc:464] Registering TS with master...
I20260927 22:59:44.090214 16127 heartbeater.cc:511] Master 127.4.74.190:33763 requested a full tablet report, sending...
I20260927 22:59:44.091773 15770 ts_manager.cc:194] Registered new tserver with Master: 69fdcfebccdb4dc4808c98771f26289a (127.4.74.130:42305)
I20260927 22:59:44.092595 15770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:53625
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 22:59:44.139817 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:54169/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:33763
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:35037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 22:59:44.439857 15985 heartbeater.cc:503] Master 127.4.74.190:33763 was elected leader, sending a full tablet report...
W20260927 22:59:44.482259 16133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 22:59:44.482617 16133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 22:59:44.482717 16133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 22:59:44.490602 16133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 22:59:44.490723 16133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 22:59:44.490841 16133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 22:59:44.501152 16133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.74.148:54169/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:33763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 22:59:44.502485 16133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 22:59:44.503820 16133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 22:59:44.518683 16138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:59:44.518719 16139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 22:59:44.519255 16141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 22:59:44.520346 16133 server_base.cc:1034] running on GCE node
I20260927 22:59:44.521091 16133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 22:59:44.522184 16133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 22:59:44.523460 16133 hybrid_clock.cc:648] HybridClock initialized: now 1790549984523399 us; error 70 us; skew 500 ppm
I20260927 22:59:44.523882 16133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 22:59:44 dist-test-slave-5pd3 krb5kdc[15294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549984, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 22:59:44.527848 16133 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 22:59:44.529699 16133 webserver.cc:460] Webserver started at http://127.4.74.131:45861/ using document root <none> and password file <none>
I20260927 22:59:44.530555 16133 fs_manager.cc:359] Metadata directory not provided
I20260927 22:59:44.530705 16133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 22:59:44.531086 16133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 22:59:44 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549984, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 22:59:44.549490 16133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "c4b29b05cac149218670717e08159444"
format_stamp: "Formatted at 2026-09-27 22:59:44 on dist-test-slave-5pd3"
server_key: "f77bb9fc3322afd6bee139fd2831c56d"
server_key_iv: "5c6035a3ba10d7e94bf71e87b78f33a5"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:44.550384 16133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "c4b29b05cac149218670717e08159444"
format_stamp: "Formatted at 2026-09-27 22:59:44 on dist-test-slave-5pd3"
server_key: "f77bb9fc3322afd6bee139fd2831c56d"
server_key_iv: "5c6035a3ba10d7e94bf71e87b78f33a5"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:44.556262 16133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260927 22:59:44.576913 16148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 22:59:44.578585 16133 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260927 22:59:44.578763 16133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "c4b29b05cac149218670717e08159444"
format_stamp: "Formatted at 2026-09-27 22:59:44 on dist-test-slave-5pd3"
server_key: "f77bb9fc3322afd6bee139fd2831c56d"
server_key_iv: "5c6035a3ba10d7e94bf71e87b78f33a5"
server_key_version: "kuduclusterkey@0"
I20260927 22:59:44.578995 16133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 22:59:44.602254 16133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 22:59:44.606483 16133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 22:59:44.606793 16133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 22:59:44.607903 16133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 22:59:44.609377 16133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 22:59:44.609455 16133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 22:59:44.609601 16133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 22:59:44.609664 16133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 22:59:44.650151 16133 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:45575
I20260927 22:59:44.650219 16260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:45575 every 8 connection(s)
I20260927 22:59:44.651721 16133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 22:59:44.660355 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 16133
I20260927 22:59:44.660567 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 22:59:44.661168 4394 external_mini_cluster.cc:1428] Setting key dd5193d6190885fc94cb13d7021bef47
Sep 27 22:59:44 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549984, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 22:59:44.677814 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.655126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:40307 (local address 127.4.74.190:33763)
0927 22:59:44.655726 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:44.655731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:44.656793 (+ 1062us) server_negotiation.cc:407] Connection header received
0927 22:59:44.659332 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:44.659338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:44.659447 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:44.659601 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:59:44.661910 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.662939 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.665637 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.666183 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.666309 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.670340 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:59:44.670347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:59:44.670351 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:59:44.670393 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:59:44.673480 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:44.674525 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:44.674530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:44.674533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:44.674683 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:44.675202 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:44.675212 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:44.675214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:44.675662 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:59:44.675836 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:44.677067 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0927 22:59:44.677389 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":305,"threads_started":1}
I20260927 22:59:44.678949 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.655697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:33763 (local address 127.4.74.131:40307)
0927 22:59:44.656582 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0927 22:59:44.656641 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 22:59:44.658953 (+ 2312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:59:44.659935 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:59:44.659951 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:59:44.660666 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0927 22:59:44.661686 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.661715 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.663195 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.663201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.665444 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.665464 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.666430 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.666436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.666658 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.667427 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 22:59:44.667456 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 22:59:44.670022 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 22:59:44.673726 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:59:44.673738 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:59:44.673760 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:59:44.674340 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:59:44.674804 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 22:59:44.674810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 22:59:44.674815 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 22:59:44.675068 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 22:59:44.675890 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 22:59:44.675899 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 22:59:44.676482 (+ 583us) client_negotiation.cc:769] Sending connection context
0927 22:59:44.676896 (+ 414us) client_negotiation.cc:240] Negotiation successful
0927 22:59:44.677510 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":333,"threads_started":1}
I20260927 22:59:44.680514 16261 heartbeater.cc:347] Connected to a master server at 127.4.74.190:33763
I20260927 22:59:44.680943 16261 heartbeater.cc:464] Registering TS with master...
I20260927 22:59:44.681874 16261 heartbeater.cc:511] Master 127.4.74.190:33763 requested a full tablet report, sending...
I20260927 22:59:44.683910 15770 ts_manager.cc:194] Registered new tserver with Master: c4b29b05cac149218670717e08159444 (127.4.74.131:45575)
I20260927 22:59:44.684845 15770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:40307
I20260927 22:59:44.691480 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 22:59:44.712591 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.696849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.4.74.190:33763)
0927 22:59:44.697093 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:44.697102 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:44.697628 (+ 526us) server_negotiation.cc:407] Connection header received
0927 22:59:44.697873 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:44.697878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:44.697977 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:44.698108 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:59:44.699769 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.700675 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.702390 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.702923 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.703083 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.706009 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:59:44.706015 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:59:44.706019 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:59:44.706059 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:59:44.709356 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:44.710279 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:44.710284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:44.710286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:44.710461 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:44.710906 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:44.710911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:44.710913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:44.711276 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:59:44.711501 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:44.712058 (+ 557us) server_negotiation.cc:299] Negotiation successful
0927 22:59:44.712301 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 22:59:44.717862 15770 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 22:59:44.725975 15770 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 22:59:44.759675 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.742367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59522 (local address 127.4.74.129:44789)
0927 22:59:44.745054 (+ 2687us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:44.745078 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:44.745124 (+ 46us) server_negotiation.cc:407] Connection header received
0927 22:59:44.745273 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:44.745282 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:44.745690 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:44.745993 (+ 303us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:59:44.747481 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.749919 (+ 2438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.756527 (+ 6608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.758205 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.758379 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.758676 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:44.759057 (+ 381us) server_negotiation.cc:299] Negotiation successful
0927 22:59:44.759260 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2168,"thread_start_us":1953,"threads_started":1}
I20260927 22:59:44.760161 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.742756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49440 (local address 127.4.74.130:42305)
0927 22:59:44.743580 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:44.743595 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:44.746828 (+ 3233us) server_negotiation.cc:407] Connection header received
0927 22:59:44.746971 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:44.746984 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:44.747386 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:44.747659 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:59:44.750302 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.752537 (+ 2235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.757069 (+ 4532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.758453 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.758578 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.758747 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:44.759654 (+ 907us) server_negotiation.cc:299] Negotiation successful
0927 22:59:44.759836 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":1173,"threads_started":1}
I20260927 22:59:44.760237 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.741932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44789 (local address 127.0.0.1:59522)
0927 22:59:44.744401 (+ 2469us) negotiation.cc:107] Waiting for socket to connect
0927 22:59:44.744469 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0927 22:59:44.744684 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:59:44.746447 (+ 1763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:59:44.746471 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:59:44.746523 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:59:44.747173 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.747192 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.750215 (+ 3023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.750223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.755435 (+ 5212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.755454 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.758468 (+ 3014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.758475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.758720 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.758780 (+ 60us) client_negotiation.cc:769] Sending connection context
0927 22:59:44.758972 (+ 192us) client_negotiation.cc:240] Negotiation successful
0927 22:59:44.759244 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2094,"mutex_wait_us":2,"spinlock_wait_cycles":3328,"thread_start_us":1017,"threads_started":1}
I20260927 22:59:44.761053 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.742714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:42305 (local address 127.0.0.1:49440)
0927 22:59:44.746451 (+ 3737us) negotiation.cc:107] Waiting for socket to connect
0927 22:59:44.746478 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 22:59:44.746645 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:59:44.747987 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:59:44.747996 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:59:44.748032 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:59:44.748618 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.748634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.752796 (+ 4162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.752806 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.756866 (+ 4060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.756880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.758865 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.758870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.759084 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.759171 (+ 87us) client_negotiation.cc:769] Sending connection context
0927 22:59:44.759277 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 22:59:44.759465 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2095,"mutex_wait_us":102,"thread_start_us":1109,"threads_started":1}
I20260927 22:59:44.764178 16278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.745181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:45575 (local address 127.0.0.1:40366)
0927 22:59:44.756092 (+ 10911us) negotiation.cc:107] Waiting for socket to connect
0927 22:59:44.756113 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 22:59:44.756254 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:59:44.756958 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:59:44.756965 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:59:44.756993 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:59:44.757408 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.757418 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.759100 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.759105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.761378 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.761392 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.763262 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.763269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.763461 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.763498 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 22:59:44.763643 (+ 145us) client_negotiation.cc:240] Negotiation successful
0927 22:59:44.763799 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10766,"mutex_wait_us":1,"thread_start_us":10609,"threads_started":1}
I20260927 22:59:44.764254 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.745302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.4.74.131:45575)
0927 22:59:44.754848 (+ 9546us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:44.754858 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:44.756263 (+ 1405us) server_negotiation.cc:407] Connection header received
0927 22:59:44.756427 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:44.756435 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:44.756722 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:44.756994 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:59:44.757609 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.758912 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.761579 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.763005 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.763146 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.763399 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:44.763710 (+ 311us) server_negotiation.cc:299] Negotiation successful
0927 22:59:44.763885 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9357,"thread_start_us":9170,"threads_started":1}
I20260927 22:59:44.765311 15919 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5d603b511848f68d36b74ba02c54f4 (DEFAULT_TABLE table=test-table [id=a9a1957540d84e219ac5fca4f69a69e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:59:44.765769 16062 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5d603b511848f68d36b74ba02c54f4 (DEFAULT_TABLE table=test-table [id=a9a1957540d84e219ac5fca4f69a69e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:59:44.767263 15919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5d603b511848f68d36b74ba02c54f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:59:44.767719 16062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5d603b511848f68d36b74ba02c54f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:59:44.769282 16196 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5d603b511848f68d36b74ba02c54f4 (DEFAULT_TABLE table=test-table [id=a9a1957540d84e219ac5fca4f69a69e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 22:59:44.771081 16196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5d603b511848f68d36b74ba02c54f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 22:59:44.779999 16280 tablet_bootstrap.cc:492] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: Bootstrap starting.
I20260927 22:59:44.784025 16280 tablet_bootstrap.cc:654] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: Neither blocks nor log segments found. Creating new log.
I20260927 22:59:44.785482 16280 log.cc:824] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: Log is configured to *not* fsync() on all Append() calls
I20260927 22:59:44.786242 16281 tablet_bootstrap.cc:492] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: Bootstrap starting.
I20260927 22:59:44.788059 16283 tablet_bootstrap.cc:492] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: Bootstrap starting.
I20260927 22:59:44.790737 16281 tablet_bootstrap.cc:654] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: Neither blocks nor log segments found. Creating new log.
I20260927 22:59:44.792075 16283 tablet_bootstrap.cc:654] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: Neither blocks nor log segments found. Creating new log.
I20260927 22:59:44.792065 16281 log.cc:824] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: Log is configured to *not* fsync() on all Append() calls
I20260927 22:59:44.792454 16280 tablet_bootstrap.cc:492] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: No bootstrap required, opened a new log
I20260927 22:59:44.792809 16280 ts_tablet_manager.cc:1397] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260927 22:59:44.793615 16283 log.cc:824] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: Log is configured to *not* fsync() on all Append() calls
I20260927 22:59:44.796989 16283 tablet_bootstrap.cc:492] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: No bootstrap required, opened a new log
I20260927 22:59:44.797118 16281 tablet_bootstrap.cc:492] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: No bootstrap required, opened a new log
I20260927 22:59:44.797355 16283 ts_tablet_manager.cc:1397] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260927 22:59:44.797425 16281 ts_tablet_manager.cc:1397] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260927 22:59:44.803336 16280 raft_consensus.cc:348] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376654ad9a1a4d51b4d24bc19a618491" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44789 } } peers { permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 } } peers { permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 } }
I20260927 22:59:44.803860 16280 raft_consensus.cc:374] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:59:44.803972 16280 raft_consensus.cc:729] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 376654ad9a1a4d51b4d24bc19a618491, State: Initialized, Role: FOLLOWER
I20260927 22:59:44.804698 16280 consensus_queue.cc:260] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376654ad9a1a4d51b4d24bc19a618491" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44789 } } peers { permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 } } peers { permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 } }
I20260927 22:59:44.806783 16280 ts_tablet_manager.cc:1428] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260927 22:59:44.807986 16281 raft_consensus.cc:348] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376654ad9a1a4d51b4d24bc19a618491" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44789 } } peers { permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 } } peers { permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 } }
I20260927 22:59:44.808729 16281 raft_consensus.cc:374] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:59:44.808856 16281 raft_consensus.cc:729] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69fdcfebccdb4dc4808c98771f26289a, State: Initialized, Role: FOLLOWER
I20260927 22:59:44.809536 16281 consensus_queue.cc:260] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376654ad9a1a4d51b4d24bc19a618491" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44789 } } peers { permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 } } peers { permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 } }
I20260927 22:59:44.810196 16283 raft_consensus.cc:348] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376654ad9a1a4d51b4d24bc19a618491" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44789 } } peers { permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 } } peers { permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 } }
I20260927 22:59:44.810758 16283 raft_consensus.cc:374] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 22:59:44.810877 16283 raft_consensus.cc:729] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b29b05cac149218670717e08159444, State: Initialized, Role: FOLLOWER
I20260927 22:59:44.810971 16127 heartbeater.cc:503] Master 127.4.74.190:33763 was elected leader, sending a full tablet report...
I20260927 22:59:44.811514 16283 consensus_queue.cc:260] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376654ad9a1a4d51b4d24bc19a618491" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44789 } } peers { permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 } } peers { permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 } }
I20260927 22:59:44.813230 16281 ts_tablet_manager.cc:1428] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260927 22:59:44.819222 16261 heartbeater.cc:503] Master 127.4.74.190:33763 was elected leader, sending a full tablet report...
I20260927 22:59:44.820103 16283 ts_tablet_manager.cc:1428] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: Time spent starting tablet: real 0.023s user 0.015s sys 0.005s
W20260927 22:59:44.858690 15988 tablet.cc:2369] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 22:59:44.903712 16262 tablet.cc:2369] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:59:44.956635 16286 raft_consensus.cc:482] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 22:59:44.957042 16286 raft_consensus.cc:504] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376654ad9a1a4d51b4d24bc19a618491" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44789 } } peers { permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 } } peers { permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 } }
I20260927 22:59:44.959168 16286 leader_election.cc:288] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69fdcfebccdb4dc4808c98771f26289a (127.4.74.130:42305), c4b29b05cac149218670717e08159444 (127.4.74.131:45575)
I20260927 22:59:44.967404 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.959935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:53597 (local address 127.4.74.131:45575)
0927 22:59:44.960176 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:44.960185 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:44.961268 (+ 1083us) server_negotiation.cc:407] Connection header received
0927 22:59:44.961522 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:44.961526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:44.961597 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:44.961705 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:59:44.962420 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.963211 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.965437 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.966267 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.966480 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.966567 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:44.966939 (+ 372us) server_negotiation.cc:299] Negotiation successful
0927 22:59:44.967077 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260927 22:59:44.967448 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.959747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:59243 (local address 127.4.74.130:42305)
0927 22:59:44.960203 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:44.960213 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:44.961122 (+ 909us) server_negotiation.cc:407] Connection header received
0927 22:59:44.961385 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:44.961390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:44.961461 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:44.961576 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 22:59:44.962351 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.963162 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.965496 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.966348 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.966569 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.966656 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:44.967002 (+ 346us) server_negotiation.cc:299] Negotiation successful
0927 22:59:44.967143 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260927 22:59:44.967689 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.959727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:42305 (local address 127.4.74.129:59243)
0927 22:59:44.960930 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0927 22:59:44.960968 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 22:59:44.961197 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:59:44.961614 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:59:44.961619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:59:44.961663 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:59:44.962168 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.962185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.963319 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.963326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.965332 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.965346 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.966614 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.966621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.966820 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.966850 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 22:59:44.966966 (+ 116us) client_negotiation.cc:240] Negotiation successful
0927 22:59:44.967133 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"mutex_wait_us":15,"spinlock_wait_cycles":2304,"thread_start_us":742,"threads_started":1}
I20260927 22:59:44.968081 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:44.959911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:45575 (local address 127.4.74.129:53597)
0927 22:59:44.961113 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0927 22:59:44.961132 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 22:59:44.961347 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 22:59:44.961722 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 22:59:44.961731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 22:59:44.961755 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 22:59:44.962238 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.962249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.963358 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.963365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.965273 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 22:59:44.965285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 22:59:44.966505 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 22:59:44.966511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 22:59:44.966753 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:44.966801 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 22:59:44.966912 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 22:59:44.967068 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"mutex_wait_us":94,"thread_start_us":595,"threads_started":1}
I20260927 22:59:44.968910 16082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5d603b511848f68d36b74ba02c54f4" candidate_uuid: "376654ad9a1a4d51b4d24bc19a618491" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69fdcfebccdb4dc4808c98771f26289a" is_pre_election: true
I20260927 22:59:44.969331 16216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5d603b511848f68d36b74ba02c54f4" candidate_uuid: "376654ad9a1a4d51b4d24bc19a618491" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b29b05cac149218670717e08159444" is_pre_election: true
I20260927 22:59:44.969586 16082 raft_consensus.cc:2454] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 376654ad9a1a4d51b4d24bc19a618491 in term 0.
I20260927 22:59:44.969908 16216 raft_consensus.cc:2454] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 376654ad9a1a4d51b4d24bc19a618491 in term 0.
I20260927 22:59:44.970512 15875 leader_election.cc:302] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 376654ad9a1a4d51b4d24bc19a618491, 69fdcfebccdb4dc4808c98771f26289a; no voters:
I20260927 22:59:44.971066 16286 raft_consensus.cc:2781] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 22:59:44.971239 16286 raft_consensus.cc:482] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 22:59:44.971377 16286 raft_consensus.cc:3037] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:59:44.973922 16286 raft_consensus.cc:504] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376654ad9a1a4d51b4d24bc19a618491" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44789 } } peers { permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 } } peers { permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 } }
I20260927 22:59:44.974720 16286 leader_election.cc:288] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [CANDIDATE]: Term 1 election: Requested vote from peers 69fdcfebccdb4dc4808c98771f26289a (127.4.74.130:42305), c4b29b05cac149218670717e08159444 (127.4.74.131:45575)
I20260927 22:59:44.975401 16216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5d603b511848f68d36b74ba02c54f4" candidate_uuid: "376654ad9a1a4d51b4d24bc19a618491" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b29b05cac149218670717e08159444"
I20260927 22:59:44.975402 16082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5d603b511848f68d36b74ba02c54f4" candidate_uuid: "376654ad9a1a4d51b4d24bc19a618491" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69fdcfebccdb4dc4808c98771f26289a"
I20260927 22:59:44.975680 16216 raft_consensus.cc:3037] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444 [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:59:44.975680 16082 raft_consensus.cc:3037] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a [term 0 FOLLOWER]: Advancing to term 1
I20260927 22:59:44.979343 16082 raft_consensus.cc:2454] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 376654ad9a1a4d51b4d24bc19a618491 in term 1.
I20260927 22:59:44.979348 16216 raft_consensus.cc:2454] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 376654ad9a1a4d51b4d24bc19a618491 in term 1.
I20260927 22:59:44.980001 15875 leader_election.cc:302] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 376654ad9a1a4d51b4d24bc19a618491, 69fdcfebccdb4dc4808c98771f26289a; no voters:
I20260927 22:59:44.980430 16286 raft_consensus.cc:2781] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 22:59:44.981094 16286 raft_consensus.cc:686] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 1 LEADER]: Becoming Leader. State: Replica: 376654ad9a1a4d51b4d24bc19a618491, State: Running, Role: LEADER
I20260927 22:59:44.981920 16286 consensus_queue.cc:237] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376654ad9a1a4d51b4d24bc19a618491" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44789 } } peers { permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 } } peers { permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 } }
I20260927 22:59:44.988972 15769 catalog_manager.cc:5495] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 reported cstate change: term changed from 0 to 1, leader changed from <none> to 376654ad9a1a4d51b4d24bc19a618491 (127.4.74.129). New cstate: current_term: 1 leader_uuid: "376654ad9a1a4d51b4d24bc19a618491" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "376654ad9a1a4d51b4d24bc19a618491" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 } health_report { overall_health: UNKNOWN } } }
I20260927 22:59:45.058625 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:45.052885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59536 (local address 127.4.74.129:44789)
0927 22:59:45.053171 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:45.053179 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:45.053202 (+ 23us) server_negotiation.cc:407] Connection header received
0927 22:59:45.053315 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:45.053319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:45.053424 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:45.053584 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 22:59:45.054035 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:45.054853 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:45.056368 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:45.056765 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:45.056896 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:45.057439 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 22:59:45.058044 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 22:59:45.058207 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:45.058262 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 22:59:45.058342 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
W20260927 22:59:45.062342 16128 tablet.cc:2369] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 22:59:45.070043 16082 raft_consensus.cc:1261] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a [term 1 FOLLOWER]: Refusing update from remote peer 376654ad9a1a4d51b4d24bc19a618491: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:59:45.070034 16216 raft_consensus.cc:1261] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444 [term 1 FOLLOWER]: Refusing update from remote peer 376654ad9a1a4d51b4d24bc19a618491: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 22:59:45.071158 16291 consensus_queue.cc:1035] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b29b05cac149218670717e08159444" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 45575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:59:45.071594 16286 consensus_queue.cc:1035] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69fdcfebccdb4dc4808c98771f26289a" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 42305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 22:59:45.088606 16295 mvcc.cc:205] Tried to move back new op lower bound from 7334092738834259968 to 7334092738493063168. Current Snapshot: MvccSnapshot[applied={T|T < 7334092738834259968}]
I20260927 22:59:45.105746 15770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:59:45.106103 15770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 22:59:45.110720 15770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0da76da7e9a143fa962fc02b5c7a3b2f: Sending DeleteTablet for 3 replicas of tablet 2b5d603b511848f68d36b74ba02c54f4
I20260927 22:59:45.112317 15919 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b5d603b511848f68d36b74ba02c54f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:59:45 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:59522
I20260927 22:59:45.112308 16196 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b5d603b511848f68d36b74ba02c54f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:59:45 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:40366
I20260927 22:59:45.113282 16062 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b5d603b511848f68d36b74ba02c54f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 22:59:45 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:49440
I20260927 22:59:45.114390 16305 tablet_replica.cc:330] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: stopping tablet replica
I20260927 22:59:45.114928 16305 raft_consensus.cc:2229] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:59:45.115578 16305 raft_consensus.cc:2258] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:59:45.117908 16305 ts_tablet_manager.cc:1905] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:59:45.119899 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 15851
I20260927 22:59:45.125530 16305 ts_tablet_manager.cc:1918] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:59:45.125833 16305 log.cc:1196] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/2b5d603b511848f68d36b74ba02c54f4
I20260927 22:59:45.126458 16305 ts_tablet_manager.cc:1939] T 2b5d603b511848f68d36b74ba02c54f4 P 69fdcfebccdb4dc4808c98771f26289a: Deleting consensus metadata
I20260927 22:59:45.128599 15757 catalog_manager.cc:4841] TS 69fdcfebccdb4dc4808c98771f26289a (127.4.74.130:42305): tablet 2b5d603b511848f68d36b74ba02c54f4 (table test-table [id=a9a1957540d84e219ac5fca4f69a69e7]) successfully deleted
I20260927 22:59:45.129056 16303 tablet_replica.cc:330] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: stopping tablet replica
I20260927 22:59:45.129853 16303 raft_consensus.cc:2229] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 22:59:45.130671 16303 raft_consensus.cc:2258] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:59:45.133957 16303 ts_tablet_manager.cc:1905] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:59:45.138147 16304 tablet_replica.cc:330] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: stopping tablet replica
I20260927 22:59:45.138722 16304 raft_consensus.cc:2229] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 1 LEADER]: Raft consensus shutting down.
I20260927 22:59:45.139537 16304 raft_consensus.cc:2258] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 22:59:45.141965 16304 ts_tablet_manager.cc:1905] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 22:59:45.143283 16303 ts_tablet_manager.cc:1918] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:59:45.143532 16303 log.cc:1196] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/2b5d603b511848f68d36b74ba02c54f4
I20260927 22:59:45.144068 16303 ts_tablet_manager.cc:1939] T 2b5d603b511848f68d36b74ba02c54f4 P c4b29b05cac149218670717e08159444: Deleting consensus metadata
I20260927 22:59:45.146116 15755 catalog_manager.cc:4841] TS c4b29b05cac149218670717e08159444 (127.4.74.131:45575): tablet 2b5d603b511848f68d36b74ba02c54f4 (table test-table [id=a9a1957540d84e219ac5fca4f69a69e7]) successfully deleted
Sep 27 22:59:45 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 22:59:45.150777 16304 ts_tablet_manager.cc:1918] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 22:59:45.151024 16304 log.cc:1196] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/2b5d603b511848f68d36b74ba02c54f4
I20260927 22:59:45.151775 16304 ts_tablet_manager.cc:1939] T 2b5d603b511848f68d36b74ba02c54f4 P 376654ad9a1a4d51b4d24bc19a618491: Deleting consensus metadata
I20260927 22:59:45.154110 15754 catalog_manager.cc:4841] TS 376654ad9a1a4d51b4d24bc19a618491 (127.4.74.129:44789): tablet 2b5d603b511848f68d36b74ba02c54f4 (table test-table [id=a9a1957540d84e219ac5fca4f69a69e7]) successfully deleted
I20260927 22:59:45.159166 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:45.121784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59548 (local address 127.4.74.129:44789)
0927 22:59:45.122089 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:45.122100 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:45.123292 (+ 1192us) server_negotiation.cc:407] Connection header received
0927 22:59:45.123604 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:45.123614 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:45.123764 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:45.123899 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:59:45.126934 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:45.128770 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:45.138596 (+ 9826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:45.139500 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:45.139752 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:45.151487 (+ 11735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:59:45.151515 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:59:45.151535 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:59:45.151657 (+ 122us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:59:45.155748 (+ 4091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:45.156659 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:45.156670 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:45.156680 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:45.156862 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:45.157291 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:45.157296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:45.157299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:45.157737 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:59:45.158105 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:45.158441 (+ 336us) server_negotiation.cc:299] Negotiation successful
0927 22:59:45.158754 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260927 22:59:45.160135 15980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:59:45.268617 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 15851
I20260927 22:59:45.288130 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 15992
Sep 27 22:59:45 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 22:59:45.305729 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:45.289239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55260 (local address 127.4.74.130:42305)
0927 22:59:45.289474 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:45.289481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:45.289504 (+ 23us) server_negotiation.cc:407] Connection header received
0927 22:59:45.289574 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:45.289580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:45.289679 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:45.289809 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:59:45.291245 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:45.292072 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:45.293427 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:45.293828 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:45.293963 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:45.298991 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:59:45.299008 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:59:45.299025 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:59:45.299064 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:59:45.302559 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:45.303378 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:45.303386 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:45.303388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:45.303522 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:45.304070 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:45.304077 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:45.304080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:45.304484 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:59:45.304770 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:45.305115 (+ 345us) server_negotiation.cc:299] Negotiation successful
0927 22:59:45.305347 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260927 22:59:45.306663 16122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:59:45.409881 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 15992
I20260927 22:59:45.427515 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 16133
Sep 27 22:59:45 dist-test-slave-5pd3 krb5kdc[15294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 22:59:45.446273 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 22:59:45.428650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.4.74.131:45575)
0927 22:59:45.428834 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 22:59:45.428839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 22:59:45.428893 (+ 54us) server_negotiation.cc:407] Connection header received
0927 22:59:45.429089 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 22:59:45.429093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 22:59:45.429164 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 22:59:45.429277 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0927 22:59:45.430782 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:45.431592 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:45.433272 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 22:59:45.433689 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 22:59:45.433812 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 22:59:45.439580 (+ 5768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 22:59:45.439611 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 22:59:45.439634 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 22:59:45.439682 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 22:59:45.442851 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:45.443871 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:45.443881 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:45.443886 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:45.444065 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 22:59:45.444695 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 22:59:45.444715 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 22:59:45.444718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 22:59:45.445113 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 22:59:45.445352 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0927 22:59:45.445777 (+ 425us) server_negotiation.cc:299] Negotiation successful
0927 22:59:45.445962 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 22:59:45.447008 16256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:59:45.544814 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 16133
I20260927 22:59:45.563235 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 15738
I20260927 22:59:45.564831 15800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 22:59:45.695268 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 15738
I20260927 22:59:45.722677 4394 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 22:59:46.070472 4394 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 22:59:46.070761 4394 mini_ranger.cc:66] Stopping Ranger...
I20260927 22:59:46.417932 4394 mini_ranger.cc:68] Stopped Ranger
2026-09-27 22:59:46.418 UTC [15324] LOG: received smart shutdown request
2026-09-27 22:59:46.422 UTC [15324] LOG: background worker "logical replication launcher" (PID 15333) exited with exit code 1
2026-09-27 22:59:46.422 UTC [15328] LOG: shutting down
2026-09-27 22:59:46.623 UTC [15324] LOG: database system is shut down
2026-09-27T22:59:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (56896 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 22:59:46 dist-test-slave-5pd3 krb5kdc[16319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 22:59:46 dist-test-slave-5pd3 krb5kdc[16319](info): set up 2 sockets
Sep 27 22:59:46 dist-test-slave-5pd3 krb5kdc[16319](info): commencing operation
krb5kdc: starting...
W20260927 22:59:47.924189 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 22:59:48 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790549988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T22:59:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T22:59:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/rangeradmin_127.4.74.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/rangeradmin_127.4.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/rangerlookup_127.4.74.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/rangerlookup_127.4.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/HTTP_127.4.74.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/HTTP_127.4.74.148@KRBTEST.COM.keytab.
I20260927 22:59:48.323446 4394 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskEVsig8/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/postgres -l logfile start
2026-09-27 22:59:51.266 UTC [16349] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 22:59:51.266 UTC [16349] LOG: listening on IPv4 address "127.4.74.148", port 47887
2026-09-27 22:59:51.270 UTC [16349] LOG: listening on Unix socket "/tmp/.s.PGSQL.47887"
2026-09-27 22:59:51.294 UTC [16352] LOG: database system was shut down at 2026-09-27 22:59:49 UTC
2026-09-27 22:59:51.298 UTC [16349] LOG: database system is ready to accept connections
I20260927 22:59:52.283581 4394 mini_postgres.cc:96] Postgres bound to 47887
2026-09-27 22:59:52.303 UTC [16360] FATAL: database "slave" does not exist
127.4.74.148:47887 - accepting connections
I20260927 22:59:52.304427 4394 mini_ranger.cc:161] Starting Ranger...
I20260927 22:59:52.334900 4394 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 22:59:52.918936 4394 mini_ranger.cc:87] Created ranger Postgres database
I20260927 22:59:52.919109 4394 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-admin
2026-09-27 22:59:53,224 [I] DB FLAVOR :POSTGRES
2026-09-27 22:59:53,224 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 22:59:53,224 [I] Checking connection..
2026-09-27 22:59:53,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 22:59:53,480 [I] Checking connection passed.
2026-09-27 22:59:53,480 [I] --------- Verifying version history table ---------
2026-09-27 22:59:53,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 22:59:53,742 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 22:59:53,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 22:59:54,023 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 22:59:54,023 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 22:59:54,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 22:59:54.296 UTC [16424] WARNING: there is no transaction in progress
2026-09-27 22:59:54,302 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 22:59:54,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 22:59:54,581 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 22:59:54,581 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 22:59:54,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 22:59:54,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 22:59:55,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5pd3.c.gcp-upstream.internal','N') ;"
2026-09-27 22:59:55,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 22:59:55,742 [I] Table x_portal_user does not exist in database ranger
2026-09-27 22:59:55,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 22:59:56,027 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 22:59:56,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 22:59:56,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 22:59:56,563 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 22:59:56,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 22:59:57.676 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.695 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.713 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.730 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.748 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.767 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.831 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.839 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.857 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.868 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.880 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.896 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.907 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.914 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.923 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:57.934 UTC [16532] WARNING: there is no transaction in progress
2026-09-27 22:59:58,763 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 22:59:58,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 22:59:59,028 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 22:59:59,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 22:59:59,292 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 22:59:59,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 22:59:59,594 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 22:59:59,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 22:59:59,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 23:00:00,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5pd3.c.gcp-upstream.internal';"
2026-09-27 23:00:00,450 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 23:00:00,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 23:00:00,711 [I] DB_PATCHES have already been applied
I20260927 23:00:00.718405 4394 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-admin:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskEVsig8/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 23:00:00.718508 4394 mini_ranger.cc:193] Using host: 127.4.74.148
I20260927 23:00:00.735934 4394 mini_ranger.cc:227] Ranger admin URL: http://127.4.74.148:57415
Sep 27, 2026 11:00:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 11:00:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 11:00:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 11:00:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 11:00:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 11:00:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 11:00:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 23:00:03.201519 4394 mini_ranger.cc:160] Time spent starting Ranger: real 10.897s user 0.005s sys 0.054s
Sep 27, 2026 11:00:24 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 11:00:25 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 27, 2026 11:00:25 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 11:00:25 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 11:00:26 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 23:00:26.854167 4394 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/rangerkms_127.4.74.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/rangerkms_127.4.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.74.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.74.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.74.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/HTTP_127.4.74.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.74.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/HTTP_127.4.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 23:00:27 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550027, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 23:00:27.059371 4394 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 23:00:27.085106 4394 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 23:00:28.165910 4394 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 23:00:28.166105 4394 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-kms
I20260927 23:00:28.166168 4394 mini_ranger_kms.cc:227] Using postgres at 127.4.74.148:47887
2026-09-27 23:00:28,596 [I] DB FLAVOR :POSTGRES
2026-09-27 23:00:28,596 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 23:00:28,596 [I] Checking connection
2026-09-27 23:00:28,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 23:00:28,844 [I] connection success
2026-09-27 23:00:28,845 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 23:00:28,845 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 23:00:28,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 23:00:29,121 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 23:00:29,122 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 23:00:29,122 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 23:00:29,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.74.148:47887/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 23:00:29,443 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 23:00:30.035440 4394 mini_ranger_kms.cc:310] Created kms service
I20260927 23:00:30.287351 4394 mini_ranger_kms.cc:326] Created kudu user
I20260927 23:00:30.344730 4394 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 23:00:30 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550027, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:30.795233 4394 mini_ranger_kms.cc:384] Added ranger policy
I20260927 23:00:30.795625 4394 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-kms:/tmp/dist-test-taskEVsig8/build/asan/bin/postgresql.jar:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskEVsig8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 23:00:30.795698 4394 mini_ranger_kms.cc:240] Using host: 127.4.74.148
I20260927 23:00:30.815254 4394 mini_ranger_kms.cc:276] Ranger KMS PID: 16714
I20260927 23:00:30.815438 4394 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.74.148:60057
Sep 27, 2026 11:00:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 11:00:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 11:00:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 11:00:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 11:00:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 11:00:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 27 23:00:33 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790550033, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.74.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:33.264290 4394 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.205s user 0.125s sys 0.332s
I20260927 23:00:33.264604 4394 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 23:00:33.264803 4394 mini_ranger_kms.cc:401] 127.4.74.148:60057/kms/v1/keys
Sep 27 23:00:33 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550033, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.74.148@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
Sep 27 23:00:34 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550033, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.74.148@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
Sep 27 23:00:36 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:00:36.808091 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:34461
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:34461
--ranger_config_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:60057/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:00:37.165342 16764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:37.165670 16764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:37.165819 16764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:37.173561 16764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:00:37.173696 16764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:37.173743 16764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:37.173833 16764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:00:37.173913 16764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:00:37.184598 16764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.74.148:60057/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:34461
--ranger_config_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:34461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:37.185998 16764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:37.187431 16764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:37.201541 16769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:37.201684 16770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:37.201728 16772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:37.202427 16764 server_base.cc:1034] running on GCE node
I20260927 23:00:37.203397 16764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:37.205111 16764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:37.206501 16764 hybrid_clock.cc:648] HybridClock initialized: now 1790550037206427 us; error 87 us; skew 500 ppm
I20260927 23:00:37.207183 16764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:37 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550037, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:37.211122 16764 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:00:37.213073 16764 webserver.cc:460] Webserver started at http://127.4.74.190:46255/ using document root <none> and password file <none>
I20260927 23:00:37.213806 16764 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:37.213943 16764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:37.214267 16764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 23:00:37 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550037, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:37.286221 16764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "c2aaf41e46ff4a449e03fff1d9f99d24"
format_stamp: "Formatted at 2026-09-27 23:00:37 on dist-test-slave-5pd3"
server_key: "360bed0332e1fbab765467ef1c5e34ae"
server_key_iv: "b4ebdbc672b29addeb15900f6ae9d5c3"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:37.287315 16764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "c2aaf41e46ff4a449e03fff1d9f99d24"
format_stamp: "Formatted at 2026-09-27 23:00:37 on dist-test-slave-5pd3"
server_key: "360bed0332e1fbab765467ef1c5e34ae"
server_key_iv: "b4ebdbc672b29addeb15900f6ae9d5c3"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:37.294047 16764 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:00:37.316658 16779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:37.318662 16764 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:00:37.318915 16764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "c2aaf41e46ff4a449e03fff1d9f99d24"
format_stamp: "Formatted at 2026-09-27 23:00:37 on dist-test-slave-5pd3"
server_key: "360bed0332e1fbab765467ef1c5e34ae"
server_key_iv: "b4ebdbc672b29addeb15900f6ae9d5c3"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:37.319209 16764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:37.341286 16764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:37.346484 16764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:37.346921 16764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:37.370205 16764 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:34461
I20260927 23:00:37.370244 16830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:34461 every 8 connection(s)
I20260927 23:00:37.372051 16764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:00:37.376396 16831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:37.380333 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 16764
I20260927 23:00:37.380630 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:00:37.381398 4394 external_mini_cluster.cc:1428] Setting key 1c21c72918cbd1815c7e4dc536741e84
I20260927 23:00:37.389770 16831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: Bootstrap starting.
I20260927 23:00:37.393788 16831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:37.395229 16831 log.cc:824] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:37.398389 16831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: No bootstrap required, opened a new log
Sep 27 23:00:37 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:37.407312 16831 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } }
I20260927 23:00:37.407847 16831 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:37.407987 16831 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2aaf41e46ff4a449e03fff1d9f99d24, State: Initialized, Role: FOLLOWER
I20260927 23:00:37.408725 16831 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } }
I20260927 23:00:37.409010 16831 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:00:37.409173 16831 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:00:37.409379 16831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:37.410212 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:37.384172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56930 (local address 127.4.74.190:34461)
0927 23:00:37.385189 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:37.385207 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:37.385269 (+ 62us) server_negotiation.cc:407] Connection header received
0927 23:00:37.387913 (+ 2644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:37.387945 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:37.388521 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:37.388999 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:37.390724 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:37.392853 (+ 2129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:37.394657 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:37.395708 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:37.395979 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:37.400496 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:37.400506 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:37.400529 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:37.400595 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:37.404585 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:37.405565 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:37.405574 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:37.405589 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:37.405816 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:37.406298 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:37.406302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:37.406304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:37.407053 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:37.407416 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:37.407992 (+ 576us) server_negotiation.cc:299] Negotiation successful
0927 23:00:37.408643 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":440,"threads_started":1}
I20260927 23:00:37.412554 16831 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } }
I20260927 23:00:37.413096 16831 leader_election.cc:302] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2aaf41e46ff4a449e03fff1d9f99d24; no voters:
I20260927 23:00:37.413837 16831 leader_election.cc:288] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:00:37.414052 16836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:00:37.414986 16836 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 1 LEADER]: Becoming Leader. State: Replica: c2aaf41e46ff4a449e03fff1d9f99d24, State: Running, Role: LEADER
I20260927 23:00:37.415822 16836 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } }
I20260927 23:00:37.416404 16831 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:00:37.419266 16838 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2aaf41e46ff4a449e03fff1d9f99d24. Latest consensus state: current_term: 1 leader_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } } }
I20260927 23:00:37.419878 16838 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:37.420687 16837 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } } }
I20260927 23:00:37.421428 16837 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:37.424407 16843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:00:37.431098 16843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:00:37.432183 16831 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 23:00:37.459707 16843 catalog_manager.cc:1347] Generated new cluster ID: d8d352a6fb6b4bd580280534296c0a58
I20260927 23:00:37.459885 16843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:00:37.512601 16843 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:00:37.514560 16843 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:00:37.523993 16843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: Generated new TSK 0
I20260927 23:00:37.525075 16843 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 23:00:38.376175 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:60057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34461
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 23:00:38 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790550038, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 23:00:38.960628 16876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:38.961161 16876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:38.961398 16876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:38.973320 16876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:38.973568 16876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:38.973816 16876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:00:38.990033 16876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.74.148:60057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:38.992051 16876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:38.993919 16876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:39.017320 16886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:39.017367 16887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:39.027247 16876 server_base.cc:1034] running on GCE node
W20260927 23:00:39.027734 16889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:39.028726 16876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:39.030224 16876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:39.031623 16876 hybrid_clock.cc:648] HybridClock initialized: now 1790550039031503 us; error 70 us; skew 500 ppm
I20260927 23:00:39.032111 16876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:39 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:39.037201 16876 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:00:39.046309 16876 webserver.cc:460] Webserver started at http://127.4.74.129:36499/ using document root <none> and password file <none>
I20260927 23:00:39.047432 16876 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:39.047643 16876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:39.048080 16876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 23:00:39 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:39.067643 16876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "00a6ea00a48b4af58712346a02cd45fb"
format_stamp: "Formatted at 2026-09-27 23:00:39 on dist-test-slave-5pd3"
server_key: "cb011074088882194e657583ff55a216"
server_key_iv: "034ef5e66d114c8e897516b1730a721b"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:39.069144 16876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "00a6ea00a48b4af58712346a02cd45fb"
format_stamp: "Formatted at 2026-09-27 23:00:39 on dist-test-slave-5pd3"
server_key: "cb011074088882194e657583ff55a216"
server_key_iv: "034ef5e66d114c8e897516b1730a721b"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:39.086295 16876 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.000s sys 0.012s
I20260927 23:00:39.108919 16896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:39.111100 16876 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260927 23:00:39.111389 16876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "00a6ea00a48b4af58712346a02cd45fb"
format_stamp: "Formatted at 2026-09-27 23:00:39 on dist-test-slave-5pd3"
server_key: "cb011074088882194e657583ff55a216"
server_key_iv: "034ef5e66d114c8e897516b1730a721b"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:39.111754 16876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:39.127907 16876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:39.134697 16876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:39.135233 16876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:39.156199 16876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:00:39.175809 16876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:00:39.176065 16876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:39.176319 16876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:00:39.176638 16876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 23:00:39 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550038, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:39.434660 16876 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:45151
I20260927 23:00:39.434697 17008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:45151 every 8 connection(s)
I20260927 23:00:39.436267 16876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:00:39.442032 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 16876
I20260927 23:00:39.442298 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 23:00:39.443207 4394 external_mini_cluster.cc:1428] Setting key e12b3a5e22a2a833644f5fa9d57f883c
Sep 27 23:00:39 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550039, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:39.473865 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:39.440816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34461 (local address 127.4.74.129:35685)
0927 23:00:39.442173 (+ 1357us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:39.442255 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:39.444924 (+ 2669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:39.447646 (+ 2722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:39.447659 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:39.448302 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:39.449152 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:39.449176 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:39.450838 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:39.450844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:39.452923 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:39.452949 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:39.454720 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:39.454729 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:39.455091 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:39.458508 (+ 3417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:39.458562 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:39.461921 (+ 3359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:39.467262 (+ 5341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:39.467277 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:39.467306 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:39.468196 (+ 890us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:39.468803 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:39.468812 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:39.468818 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:39.469141 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:39.469950 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:39.469963 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:39.470612 (+ 649us) client_negotiation.cc:769] Sending connection context
0927 23:00:39.471006 (+ 394us) client_negotiation.cc:240] Negotiation successful
0927 23:00:39.471792 (+ 786us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":795,"threads_started":1}
I20260927 23:00:39.475713 17012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:39.441582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:35685 (local address 127.4.74.190:34461)
0927 23:00:39.447001 (+ 5419us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:39.447008 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:39.447028 (+ 20us) server_negotiation.cc:407] Connection header received
0927 23:00:39.447093 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:39.447098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:39.447180 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:39.449349 (+ 2169us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:39.449520 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:39.450649 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:39.453151 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:39.454141 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:39.454315 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:39.462410 (+ 8095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:39.462422 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:39.462430 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:39.462495 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:39.466980 (+ 4485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:39.468445 (+ 1465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:39.468454 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:39.468460 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:39.468662 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:39.469317 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:39.469324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:39.469330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:39.469781 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:39.474408 (+ 4627us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:39.474716 (+ 308us) server_negotiation.cc:299] Negotiation successful
0927 23:00:39.475117 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5073,"thread_start_us":4938,"threads_started":1}
I20260927 23:00:39.478645 17009 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34461
I20260927 23:00:39.479157 17009 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 23:00:39.480224 17009 heartbeater.cc:511] Master 127.4.74.190:34461 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
I20260927 23:00:39.483444 16796 ts_manager.cc:194] Registered new tserver with Master: 00a6ea00a48b4af58712346a02cd45fb (127.4.74.129:45151)
I20260927 23:00:39.486178 16796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:35685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 23:00:39.520059 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:60057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34461
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 23:00:39 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550038, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
W20260927 23:00:40.001327 17015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:40.001747 17015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:40.001957 17015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:40.012778 17015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:40.012925 17015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:40.013103 17015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:00:40.027076 17015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.74.148:60057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:40.029353 17015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:40.032195 17015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:40.057731 17029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:40.057753 17030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:40.057952 17032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:40.059839 17015 server_base.cc:1034] running on GCE node
I20260927 23:00:40.060928 17015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:40.062523 17015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:40.063833 17015 hybrid_clock.cc:648] HybridClock initialized: now 1790550040063717 us; error 72 us; skew 500 ppm
I20260927 23:00:40.064494 17015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:40 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550040, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:40.076328 17015 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:00:40.078930 17015 webserver.cc:460] Webserver started at http://127.4.74.130:39011/ using document root <none> and password file <none>
I20260927 23:00:40.080165 17015 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:40.080348 17015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:40.080780 17015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 23:00:40 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550040, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:40.104758 17015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "45b8872725a547f98408d6f7d54b4ffe"
format_stamp: "Formatted at 2026-09-27 23:00:40 on dist-test-slave-5pd3"
server_key: "8f24076c959ae0891701318af2381811"
server_key_iv: "c7d9e54d96a641cf6cb755f243025df3"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:40.106072 17015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "45b8872725a547f98408d6f7d54b4ffe"
format_stamp: "Formatted at 2026-09-27 23:00:40 on dist-test-slave-5pd3"
server_key: "8f24076c959ae0891701318af2381811"
server_key_iv: "c7d9e54d96a641cf6cb755f243025df3"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:40.115375 17015 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s
I20260927 23:00:40.140393 17039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:40.143172 17015 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.002s
I20260927 23:00:40.143518 17015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "45b8872725a547f98408d6f7d54b4ffe"
format_stamp: "Formatted at 2026-09-27 23:00:40 on dist-test-slave-5pd3"
server_key: "8f24076c959ae0891701318af2381811"
server_key_iv: "c7d9e54d96a641cf6cb755f243025df3"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:40.144071 17015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:40.173425 17015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:40.186860 17015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:40.187287 17015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:40.188792 17015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:00:40.190761 17015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:00:40.190902 17015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260927 23:00:40.191061 17015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:00:40.191139 17015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:40.243939 17015 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:40973
I20260927 23:00:40.244036 17151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:40973 every 8 connection(s)
I20260927 23:00:40.245771 17015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:00:40.250345 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 17015
I20260927 23:00:40.250600 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 23:00:40.251276 4394 external_mini_cluster.cc:1428] Setting key a50e2d46bfb0caa33d2b1ba0d812323b
Sep 27 23:00:40 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550040, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:40.280504 17155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:40.250849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:49177 (local address 127.4.74.190:34461)
0927 23:00:40.256135 (+ 5286us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:40.256144 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:40.257215 (+ 1071us) server_negotiation.cc:407] Connection header received
0927 23:00:40.257299 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:40.257804 (+ 505us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:40.257940 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:40.258107 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:40.261440 (+ 3333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:40.262815 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:40.264839 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:40.265287 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:40.265438 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:40.270793 (+ 5355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:40.270804 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:40.270811 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:40.270888 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:40.275321 (+ 4433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:40.276895 (+ 1574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:40.276904 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:40.276909 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:40.277167 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:40.277853 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:40.277859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:40.277863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:40.278335 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:40.278495 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:40.279729 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0927 23:00:40.280058 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5049,"thread_start_us":4898,"threads_started":1}
I20260927 23:00:40.282984 17154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:40.250169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34461 (local address 127.4.74.130:49177)
0927 23:00:40.251265 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:40.251339 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:40.253601 (+ 2262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:40.258838 (+ 5237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:40.258862 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:40.259854 (+ 992us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:40.261121 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:40.261158 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:40.263012 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:40.263018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:40.264630 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:40.264656 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:40.265567 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:40.265574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:40.265863 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:40.266700 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:40.266727 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:40.270412 (+ 3685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:40.275714 (+ 5302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:40.275728 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:40.275751 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:40.276651 (+ 900us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:40.277389 (+ 738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:40.277394 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:40.277398 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:40.277696 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:40.278604 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:40.278612 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:40.279176 (+ 564us) client_negotiation.cc:769] Sending connection context
0927 23:00:40.280861 (+ 1685us) client_negotiation.cc:240] Negotiation successful
0927 23:00:40.281508 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":673,"threads_started":1}
I20260927 23:00:40.285331 17152 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34461
I20260927 23:00:40.285727 17152 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:40.286675 17152 heartbeater.cc:511] Master 127.4.74.190:34461 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 23:00:40.289008 16796 ts_manager.cc:194] Registered new tserver with Master: 45b8872725a547f98408d6f7d54b4ffe (127.4.74.130:40973)
I20260927 23:00:40.290023 16796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:49177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 23:00:40.329463 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:60057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34461
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:00:40.491359 17009 heartbeater.cc:503] Master 127.4.74.190:34461 was elected leader, sending a full tablet report...
W20260927 23:00:40.746273 17158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:40.746702 17158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:40.746870 17158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:40.755208 17158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:40.755411 17158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:40.755664 17158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:00:40.767696 17158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.74.148:60057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:34461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:40.769485 17158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:40.771142 17158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:40.788573 17163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:40.788784 17164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:40.789074 17166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:40.790211 17158 server_base.cc:1034] running on GCE node
I20260927 23:00:40.791056 17158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:40.792344 17158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:40.793583 17158 hybrid_clock.cc:648] HybridClock initialized: now 1790550040793505 us; error 80 us; skew 500 ppm
I20260927 23:00:40.793964 17158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:40 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550040, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:40.797801 17158 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:00:40.799723 17158 webserver.cc:460] Webserver started at http://127.4.74.131:44685/ using document root <none> and password file <none>
I20260927 23:00:40.800515 17158 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:40.800664 17158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:40.800995 17158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 23:00:40 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550040, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:40.822968 17158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "8a20e944716a44ea8d386d3fb280397d"
format_stamp: "Formatted at 2026-09-27 23:00:40 on dist-test-slave-5pd3"
server_key: "efb683f7c4fc7f47b2b298c888e3745f"
server_key_iv: "e6120c03b10a94dff32e179482c57c32"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:40.824432 17158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "8a20e944716a44ea8d386d3fb280397d"
format_stamp: "Formatted at 2026-09-27 23:00:40 on dist-test-slave-5pd3"
server_key: "efb683f7c4fc7f47b2b298c888e3745f"
server_key_iv: "e6120c03b10a94dff32e179482c57c32"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:40.833617 17158 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.001s
I20260927 23:00:40.853087 17173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:40.855376 17158 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260927 23:00:40.855669 17158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "8a20e944716a44ea8d386d3fb280397d"
format_stamp: "Formatted at 2026-09-27 23:00:40 on dist-test-slave-5pd3"
server_key: "efb683f7c4fc7f47b2b298c888e3745f"
server_key_iv: "e6120c03b10a94dff32e179482c57c32"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:40.856011 17158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:40.875795 17158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:40.880383 17158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:40.880692 17158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:40.882030 17158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:00:40.883698 17158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:00:40.883782 17158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:40.883894 17158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:00:40.883944 17158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:40.924437 17158 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:37023
I20260927 23:00:40.924628 17285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:37023 every 8 connection(s)
I20260927 23:00:40.926188 17158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:00:40.931524 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 17158
I20260927 23:00:40.931782 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 23:00:40.932286 4394 external_mini_cluster.cc:1428] Setting key c59ca9ddeed6556d9898b2e2a2c95e75
Sep 27 23:00:40 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550040, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:40.952930 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:40.929551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:49153 (local address 127.4.74.190:34461)
0927 23:00:40.930233 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:40.930242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:40.931457 (+ 1215us) server_negotiation.cc:407] Connection header received
0927 23:00:40.933782 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:40.933787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:40.933947 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:40.934092 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:40.936644 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:40.937416 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:40.939705 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:40.940190 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:40.940307 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:40.944868 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:40.944873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:40.944877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:40.944921 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:40.948118 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:40.949355 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:40.949360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:40.949363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:40.949527 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:40.950193 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:40.950199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:40.950202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:40.950747 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:40.950949 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:40.952278 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0927 23:00:40.952606 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":299,"threads_started":1}
I20260927 23:00:40.953517 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:40.930255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34461 (local address 127.4.74.131:49153)
0927 23:00:40.931243 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:40.931302 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:40.933445 (+ 2143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:40.934595 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:40.934612 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:40.935295 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:40.936376 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:40.936405 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:40.937638 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:40.937644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:40.939393 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:40.939426 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:40.940503 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:40.940510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:40.940840 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:40.941730 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:40.941768 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:40.944515 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:40.948320 (+ 3805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:40.948333 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:40.948357 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:40.949202 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:40.949649 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:40.949655 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:40.949659 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:40.949878 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:40.950927 (+ 1049us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:40.950934 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:40.951428 (+ 494us) client_negotiation.cc:769] Sending connection context
0927 23:00:40.951783 (+ 355us) client_negotiation.cc:240] Negotiation successful
0927 23:00:40.952271 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":395,"threads_started":1}
I20260927 23:00:40.955597 17286 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34461
I20260927 23:00:40.956004 17286 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:40.956823 17286 heartbeater.cc:511] Master 127.4.74.190:34461 requested a full tablet report, sending...
I20260927 23:00:40.958786 16796 ts_manager.cc:194] Registered new tserver with Master: 8a20e944716a44ea8d386d3fb280397d (127.4.74.131:37023)
I20260927 23:00:40.959878 16796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:49153
I20260927 23:00:40.963901 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:00:40.965749 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 16876
Sep 27 23:00:40 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 23:00:40.986092 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:40.967031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60506 (local address 127.4.74.129:45151)
0927 23:00:40.967827 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:40.967836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:40.967872 (+ 36us) server_negotiation.cc:407] Connection header received
0927 23:00:40.967951 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:40.967959 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:40.968240 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:40.968405 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:40.970072 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:40.971369 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:40.973078 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:40.973757 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:40.973902 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:40.978955 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:40.978971 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:40.978991 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:40.979049 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:40.982566 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:40.983627 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:40.983638 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:40.983644 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:40.983853 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:40.984310 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:40.984314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:40.984317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:40.984761 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:40.985028 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:40.985404 (+ 376us) server_negotiation.cc:299] Negotiation successful
0927 23:00:40.985648 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":339,"threads_started":1}
I20260927 23:00:40.987960 17004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:41.085224 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 16876
I20260927 23:00:41.102352 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 17015
Sep 27 23:00:41 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 23:00:41.122684 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:41.103476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40084 (local address 127.4.74.130:40973)
0927 23:00:41.104220 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:41.104228 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:41.104265 (+ 37us) server_negotiation.cc:407] Connection header received
0927 23:00:41.104346 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:41.104353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:41.104642 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:41.104810 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:41.106326 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:41.107728 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:41.109207 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:41.109928 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:41.110081 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:41.115019 (+ 4938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:41.115043 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:41.115077 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:41.115232 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:41.119074 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:41.120137 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:41.120147 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:41.120150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:41.120354 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:41.120835 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:41.120840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:41.120842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:41.121330 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:41.121619 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:41.121973 (+ 354us) server_negotiation.cc:299] Negotiation successful
0927 23:00:41.122312 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":431,"threads_started":1}
I20260927 23:00:41.124687 17147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:41.219288 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 17015
I20260927 23:00:41.242559 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 17158
Sep 27 23:00:41 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 23:00:41.263353 17295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:41.243743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57134 (local address 127.4.74.131:37023)
0927 23:00:41.244482 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:41.244491 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:41.244526 (+ 35us) server_negotiation.cc:407] Connection header received
0927 23:00:41.244673 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:41.244681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:41.244947 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:41.245129 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:41.246977 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:41.248326 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:41.250047 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:41.250650 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:41.250772 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:41.256194 (+ 5422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:41.256208 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:41.256224 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:41.256281 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:41.259360 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:41.260512 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:41.260520 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:41.260523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:41.260713 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:41.261310 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:41.261314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:41.261317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:41.261788 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:41.262174 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:41.262629 (+ 455us) server_negotiation.cc:299] Negotiation successful
0927 23:00:41.262928 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":339,"threads_started":1}
I20260927 23:00:41.265038 17281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:41.362135 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 17158
I20260927 23:00:41.378875 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 16764
I20260927 23:00:41.380568 16826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:41.511327 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 16764
I20260927 23:00:41.539911 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:34461
--webserver_interface=127.4.74.190
--webserver_port=46255
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:34461
--ranger_config_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:60057/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.74.148:60057/kms with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:00:41.904229 17298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:41.904547 17298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:41.904706 17298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:41.912108 17298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:00:41.912246 17298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:41.912292 17298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:41.912417 17298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:00:41.912501 17298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:00:41.923480 17298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.74.148:60057/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:34461
--ranger_config_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:34461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=46255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:41.924949 17298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:41.926401 17298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:41.939445 17303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:41.939579 17304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:41.940053 17306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:41.941035 17298 server_base.cc:1034] running on GCE node
I20260927 23:00:41.942019 17298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:41.943778 17298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:41.945041 17298 hybrid_clock.cc:648] HybridClock initialized: now 1790550041944990 us; error 65 us; skew 500 ppm
I20260927 23:00:41.945469 17298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:41 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550041, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:41.949134 17298 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:00:41.951040 17298 webserver.cc:460] Webserver started at http://127.4.74.190:46255/ using document root <none> and password file <none>
I20260927 23:00:41.951870 17298 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:41.952024 17298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 23:00:41 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550041, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:42.023056 17298 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.001s
I20260927 23:00:42.028301 17314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:42.030867 17298 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260927 23:00:42.031303 17298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "c2aaf41e46ff4a449e03fff1d9f99d24"
format_stamp: "Formatted at 2026-09-27 23:00:37 on dist-test-slave-5pd3"
server_key: "360bed0332e1fbab765467ef1c5e34ae"
server_key_iv: "b4ebdbc672b29addeb15900f6ae9d5c3"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:42.033067 17298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:42.061100 17298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:42.066186 17298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:42.066510 17298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:42.088654 17298 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:34461
I20260927 23:00:42.088686 17365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:34461 every 8 connection(s)
I20260927 23:00:42.090533 17298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:00:42.097970 17366 sys_catalog.cc:263] Verifying existing consensus state
I20260927 23:00:42.100598 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 17298
I20260927 23:00:42.101953 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:60057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:45151
--local_ip_for_outbound_sockets=127.4.74.129
--tserver_master_addrs=127.4.74.190:34461
--webserver_port=36499
--webserver_interface=127.4.74.129
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.74.148:60057/kms with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:00:42.103746 17366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: Bootstrap starting.
I20260927 23:00:42.130589 17366 log.cc:824] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:42.142380 17366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 23:00:42.143051 17366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: Bootstrap complete.
I20260927 23:00:42.153343 17366 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } }
I20260927 23:00:42.154421 17366 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2aaf41e46ff4a449e03fff1d9f99d24, State: Initialized, Role: FOLLOWER
I20260927 23:00:42.155284 17366 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } }
I20260927 23:00:42.155613 17366 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:00:42.155762 17366 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:00:42.155974 17366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 1 FOLLOWER]: Advancing to term 2
I20260927 23:00:42.160741 17366 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } }
I20260927 23:00:42.161407 17366 leader_election.cc:302] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2aaf41e46ff4a449e03fff1d9f99d24; no voters:
I20260927 23:00:42.162211 17366 leader_election.cc:288] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 23:00:42.162860 17370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 23:00:42.164037 17370 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [term 2 LEADER]: Becoming Leader. State: Replica: c2aaf41e46ff4a449e03fff1d9f99d24, State: Running, Role: LEADER
I20260927 23:00:42.164899 17370 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } }
I20260927 23:00:42.165140 17366 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:00:42.168763 17372 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2aaf41e46ff4a449e03fff1d9f99d24. Latest consensus state: current_term: 2 leader_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } } }
I20260927 23:00:42.169219 17372 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:42.168836 17371 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2aaf41e46ff4a449e03fff1d9f99d24" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 34461 } } }
I20260927 23:00:42.169392 17371 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24 [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:42.181033 17375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:00:42.189468 17375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:00:42.191372 17375 catalog_manager.cc:1259] Loaded cluster ID: d8d352a6fb6b4bd580280534296c0a58
I20260927 23:00:42.191663 17375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:00:42.196805 17375 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:00:42.199447 17375 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: Loaded TSK: 0
I20260927 23:00:42.201965 17375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:00:42.206550 17366 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260927 23:00:42.542155 17368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:42.542656 17368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:42.542855 17368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:42.554106 17368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:42.554373 17368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:42.554622 17368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:00:42.571102 17368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.74.148:60057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:45151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=36499
--tserver_master_addrs=127.4.74.190:34461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:42.573189 17368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:42.575027 17368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:42.616277 17404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:42.618214 17403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:42.621428 17406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:42.622396 17368 server_base.cc:1034] running on GCE node
I20260927 23:00:42.623229 17368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:42.624455 17368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:42.627460 17368 hybrid_clock.cc:648] HybridClock initialized: now 1790550042627336 us; error 75 us; skew 500 ppm
I20260927 23:00:42.627951 17368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:42 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550042, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:42.638692 17368 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:00:42.650207 17368 webserver.cc:460] Webserver started at http://127.4.74.129:36499/ using document root <none> and password file <none>
I20260927 23:00:42.651592 17368 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:42.651893 17368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 23:00:42 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550042, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:42.700484 17368 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.005s sys 0.004s
I20260927 23:00:42.706804 17414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:42.709654 17368 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260927 23:00:42.710089 17368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "00a6ea00a48b4af58712346a02cd45fb"
format_stamp: "Formatted at 2026-09-27 23:00:39 on dist-test-slave-5pd3"
server_key: "cb011074088882194e657583ff55a216"
server_key_iv: "034ef5e66d114c8e897516b1730a721b"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:42.712095 17368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:42.744602 17368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:42.751706 17368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:42.752209 17368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:42.758720 17368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:00:42.761665 17368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:00:42.761955 17368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260927 23:00:42.762192 17368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:00:42.762348 17368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:42.912523 17368 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:45151
I20260927 23:00:42.912577 17527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:45151 every 8 connection(s)
I20260927 23:00:42.914669 17368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:00:42.924978 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 17368
I20260927 23:00:42.925822 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:60057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:40973
--local_ip_for_outbound_sockets=127.4.74.130
--tserver_master_addrs=127.4.74.190:34461
--webserver_port=39011
--webserver_interface=127.4.74.130
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.74.148:60057/kms with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 23:00:42 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550042, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:42.954700 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:42.918696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:33255 (local address 127.4.74.190:34461)
0927 23:00:42.919648 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:42.919673 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:42.920322 (+ 649us) server_negotiation.cc:407] Connection header received
0927 23:00:42.923052 (+ 2730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:42.923065 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:42.923473 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:42.923932 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:42.926566 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:42.928994 (+ 2428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:42.932012 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:42.933196 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:42.933419 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:42.941024 (+ 7605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:42.941039 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:42.941064 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:42.941123 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:42.946238 (+ 5115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:42.947931 (+ 1693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:42.947944 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:42.947955 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:42.948280 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:42.949088 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:42.949099 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:42.949106 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:42.950336 (+ 1230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:42.950663 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:42.952410 (+ 1747us) server_negotiation.cc:299] Negotiation successful
0927 23:00:42.952975 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":347,"threads_started":1}
I20260927 23:00:42.962055 17531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:42.919017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34461 (local address 127.4.74.129:33255)
0927 23:00:42.920125 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:42.920174 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:42.922345 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:42.924092 (+ 1747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:42.924104 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:42.924885 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:42.925785 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:42.925815 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:42.929249 (+ 3434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:42.929259 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:42.931765 (+ 2506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:42.931788 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:42.933903 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:42.933913 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:42.934254 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:42.936448 (+ 2194us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:42.936485 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:42.940589 (+ 4104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:42.946585 (+ 5996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:42.946603 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:42.946630 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:42.947604 (+ 974us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:42.948482 (+ 878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:42.948488 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:42.948492 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:42.948849 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:42.950868 (+ 2019us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:42.950884 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:42.951613 (+ 729us) client_negotiation.cc:769] Sending connection context
0927 23:00:42.953998 (+ 2385us) client_negotiation.cc:240] Negotiation successful
0927 23:00:42.954879 (+ 881us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":492,"threads_started":1}
I20260927 23:00:42.967410 17528 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34461
I20260927 23:00:42.967866 17528 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:42.969061 17528 heartbeater.cc:511] Master 127.4.74.190:34461 requested a full tablet report, sending...
W20260927 23:00:42.973701 17528 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:34461 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 23:00:42.973999 17528 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:42.974364 17528 heartbeater.cc:511] Master 127.4.74.190:34461 requested a full tablet report, sending...
I20260927 23:00:42.976784 17528 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:42.977392 17528 heartbeater.cc:511] Master 127.4.74.190:34461 requested a full tablet report, sending...
W20260927 23:00:42.979303 17528 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 23:00:43.349902 17532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:43.350219 17532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:43.350368 17532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:43.360265 17532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:43.360442 17532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:43.360637 17532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:00:43.374321 17532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.74.148:60057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:40973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=39011
--tserver_master_addrs=127.4.74.190:34461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:43.376049 17532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:43.377665 17532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:43.396072 17544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:43.397174 17545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:43.398023 17547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:43.399791 17532 server_base.cc:1034] running on GCE node
I20260927 23:00:43.400552 17532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:43.402451 17532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:43.403689 17532 hybrid_clock.cc:648] HybridClock initialized: now 1790550043403609 us; error 52 us; skew 500 ppm
I20260927 23:00:43.404054 17532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:43 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550043, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:43.408432 17532 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:00:43.410326 17532 webserver.cc:460] Webserver started at http://127.4.74.130:39011/ using document root <none> and password file <none>
I20260927 23:00:43.411101 17532 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:43.411263 17532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 23:00:43 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550043, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:43.447329 17532 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.003s sys 0.007s
I20260927 23:00:43.457077 17557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:43.459208 17532 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.007s
I20260927 23:00:43.459484 17532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "45b8872725a547f98408d6f7d54b4ffe"
format_stamp: "Formatted at 2026-09-27 23:00:40 on dist-test-slave-5pd3"
server_key: "8f24076c959ae0891701318af2381811"
server_key_iv: "c7d9e54d96a641cf6cb755f243025df3"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:43.461239 17532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:43.508713 17532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:43.523384 17532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:43.523887 17532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:43.536088 17532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:00:43.549316 17532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:00:43.549576 17532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260927 23:00:43.549747 17532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:00:43.549862 17532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 23:00:43 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790550043, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:43.778178 17532 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:40973
I20260927 23:00:43.778237 17671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:40973 every 8 connection(s)
I20260927 23:00:43.779788 17532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:00:43.787166 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 17532
I20260927 23:00:43.788017 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.74.148:60057/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:37023
--local_ip_for_outbound_sockets=127.4.74.131
--tserver_master_addrs=127.4.74.190:34461
--webserver_port=44685
--webserver_interface=127.4.74.131
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.74.148:60057/kms with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 23:00:43 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550043, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:43.819468 17674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:43.783963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:35689 (local address 127.4.74.190:34461)
0927 23:00:43.797418 (+ 13455us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:43.797427 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:43.797460 (+ 33us) server_negotiation.cc:407] Connection header received
0927 23:00:43.797555 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:43.797561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:43.797672 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:43.797864 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:43.800200 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:43.801625 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:43.804505 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:43.805264 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:43.805386 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:43.810382 (+ 4996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:43.810392 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:43.810400 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:43.810453 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:43.814791 (+ 4338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:43.816026 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:43.816035 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:43.816042 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:43.816219 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:43.816868 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:43.816876 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:43.816879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:43.817311 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:43.817476 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:43.818750 (+ 1274us) server_negotiation.cc:299] Negotiation successful
0927 23:00:43.819011 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13258,"thread_start_us":13072,"threads_started":1}
I20260927 23:00:43.822238 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:43.784292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34461 (local address 127.4.74.130:35689)
0927 23:00:43.785785 (+ 1493us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:43.785831 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:43.790620 (+ 4789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:43.798230 (+ 7610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:43.798247 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:43.798960 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:43.799904 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:43.799935 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:43.801876 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:43.801885 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:43.804291 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:43.804311 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:43.805795 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:43.805806 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:43.806162 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:43.807105 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:43.807149 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:43.810038 (+ 2889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:43.815024 (+ 4986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:43.815037 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:43.815063 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:43.815857 (+ 794us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:43.816395 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:43.816402 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:43.816407 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:43.816715 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:43.817583 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:43.817594 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:43.818271 (+ 677us) client_negotiation.cc:769] Sending connection context
0927 23:00:43.820091 (+ 1820us) client_negotiation.cc:240] Negotiation successful
0927 23:00:43.820738 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"thread_start_us":975,"threads_started":1}
I20260927 23:00:43.824384 17672 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34461
I20260927 23:00:43.824703 17672 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:43.825562 17672 heartbeater.cc:511] Master 127.4.74.190:34461 requested a full tablet report, sending...
I20260927 23:00:43.827816 17331 ts_manager.cc:194] Registered new tserver with Master: 45b8872725a547f98408d6f7d54b4ffe (127.4.74.130:40973)
I20260927 23:00:43.830082 17331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:35689
I20260927 23:00:43.979455 17528 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:43.980069 17528 heartbeater.cc:511] Master 127.4.74.190:34461 requested a full tablet report, sending...
I20260927 23:00:43.981581 17331 ts_manager.cc:194] Registered new tserver with Master: 00a6ea00a48b4af58712346a02cd45fb (127.4.74.129:45151)
I20260927 23:00:43.983035 17331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:33255
W20260927 23:00:44.246376 17676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:44.246819 17676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:44.246989 17676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:44.259053 17676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:44.259227 17676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:44.259430 17676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:00:44.275369 17676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.74.148:60057/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:37023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=44685
--tserver_master_addrs=127.4.74.190:34461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:44.277045 17676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:44.278410 17676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 27 23:00:44 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550043, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
W20260927 23:00:44.306450 17683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:44.313319 17676 server_base.cc:1034] running on GCE node
W20260927 23:00:44.314162 17682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:44.314771 17685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:44.315848 17676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:44.331074 17676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:44.332547 17676 hybrid_clock.cc:648] HybridClock initialized: now 1790550044332455 us; error 79 us; skew 500 ppm
I20260927 23:00:44.333180 17676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:44 dist-test-slave-5pd3 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550044, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:44.338179 17676 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:00:44.368312 17676 webserver.cc:460] Webserver started at http://127.4.74.131:44685/ using document root <none> and password file <none>
I20260927 23:00:44.369740 17676 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:44.370065 17676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 23:00:44 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550044, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:44.417734 17676 fs_manager.cc:709] Time spent opening directory manager: real 0.017s user 0.008s sys 0.008s
I20260927 23:00:44.436563 17693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:44.439188 17676 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.009s sys 0.008s
I20260927 23:00:44.439517 17676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "8a20e944716a44ea8d386d3fb280397d"
format_stamp: "Formatted at 2026-09-27 23:00:40 on dist-test-slave-5pd3"
server_key: "efb683f7c4fc7f47b2b298c888e3745f"
server_key_iv: "e6120c03b10a94dff32e179482c57c32"
server_key_version: "kuduclusterkey@0"
I20260927 23:00:44.441094 17676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:44.471274 17676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:44.476181 17676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:44.476498 17676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:44.477721 17676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:00:44.479260 17676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:00:44.479378 17676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:44.479518 17676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:00:44.479640 17676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260927 23:00:44.530396 17676 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:37023
I20260927 23:00:44.530450 17805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:37023 every 8 connection(s)
I20260927 23:00:44.532107 17676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:00:44.539381 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 17676
Sep 27 23:00:44 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550044, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:44.579912 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.535979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:46213 (local address 127.4.74.190:34461)
0927 23:00:44.552745 (+ 16766us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:44.552757 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:44.559047 (+ 6290us) server_negotiation.cc:407] Connection header received
0927 23:00:44.560423 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:44.560431 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:44.560567 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:44.560690 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:44.563046 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.564325 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.566797 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.567574 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.567681 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.572058 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:44.572077 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:44.572081 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:44.572129 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:44.575979 (+ 3850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:44.577000 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:44.577005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:44.577008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:44.577172 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:44.577712 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:44.577717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:44.577720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:44.578098 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:44.578225 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:44.579221 (+ 996us) server_negotiation.cc:299] Negotiation successful
0927 23:00:44.579469 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":16521,"spinlock_wait_cycles":11008,"thread_start_us":16787,"threads_started":1}
I20260927 23:00:44.582010 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.540734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56934 (local address 127.4.74.190:34461)
0927 23:00:44.554562 (+ 13828us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:44.554572 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:44.558985 (+ 4413us) server_negotiation.cc:407] Connection header received
0927 23:00:44.560359 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:44.560367 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:44.560555 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:44.560749 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:44.563680 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.565175 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.567398 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.568209 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.568316 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.572444 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:44.572452 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:44.572457 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:44.572516 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:44.577864 (+ 5348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:44.579001 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:44.579007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:44.579010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:44.579193 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:44.579724 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:44.579729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:44.579732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:44.580140 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:44.580558 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:44.581253 (+ 695us) server_negotiation.cc:299] Negotiation successful
0927 23:00:44.581551 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2433,"server-negotiator.queue_time_us":13655,"spinlock_wait_cycles":1408,"thread_start_us":13472,"threads_started":1}
I20260927 23:00:44.582845 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.536482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:34461 (local address 127.4.74.131:46213)
0927 23:00:44.555389 (+ 18907us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:44.555487 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:44.558877 (+ 3390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:44.561272 (+ 2395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:44.561285 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:44.561958 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:44.562804 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.562830 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.564515 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.564520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.566622 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.566639 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.568103 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.568109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.568386 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.569200 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:44.569229 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:44.571787 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:44.576182 (+ 4395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:44.576193 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:44.576213 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:44.576851 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:44.577308 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:44.577312 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:44.577315 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:44.577572 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:44.578311 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:44.578319 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:44.578760 (+ 441us) client_negotiation.cc:769] Sending connection context
0927 23:00:44.580825 (+ 2065us) client_negotiation.cc:240] Negotiation successful
0927 23:00:44.581453 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18470,"thread_start_us":18157,"threads_started":1}
I20260927 23:00:44.585566 17806 heartbeater.cc:347] Connected to a master server at 127.4.74.190:34461
I20260927 23:00:44.585927 17806 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:44.586810 17806 heartbeater.cc:511] Master 127.4.74.190:34461 requested a full tablet report, sending...
I20260927 23:00:44.588691 17330 ts_manager.cc:194] Registered new tserver with Master: 8a20e944716a44ea8d386d3fb280397d (127.4.74.131:37023)
I20260927 23:00:44.589949 17330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:46213
I20260927 23:00:44.601224 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:00:44.633214 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.610010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56944 (local address 127.4.74.190:34461)
0927 23:00:44.610252 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:44.610259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:44.612390 (+ 2131us) server_negotiation.cc:407] Connection header received
0927 23:00:44.612627 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:44.612633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:44.612740 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:44.612870 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:44.615243 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.616408 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.618530 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.619237 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.619416 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.624262 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:44.624272 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:44.624278 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:44.624354 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:44.628352 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:44.629481 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:44.629488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:44.629492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:44.629681 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:44.630195 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:44.630201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:44.630204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:44.630611 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:44.630887 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:44.632600 (+ 1713us) server_negotiation.cc:299] Negotiation successful
0927 23:00:44.632874 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260927 23:00:44.641367 17330 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:00:44.653998 17330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Sep 27 23:00:44 dist-test-slave-5pd3 krb5kdc[16319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550043, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for HTTP/127.4.74.148@KRBTEST.COM
I20260927 23:00:44.725176 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.692642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:40973 (local address 127.0.0.1:40096)
0927 23:00:44.706246 (+ 13604us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:44.706328 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:44.706646 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:44.710502 (+ 3856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:44.710512 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:44.710556 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:00:44.711086 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.711110 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.719100 (+ 7990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.719107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.721397 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.721420 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.723883 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.723893 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.724165 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.724225 (+ 60us) client_negotiation.cc:769] Sending connection context
0927 23:00:44.724403 (+ 178us) client_negotiation.cc:240] Negotiation successful
0927 23:00:44.724675 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13160,"thread_start_us":14411,"threads_started":1}
I20260927 23:00:44.727793 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.693460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40096 (local address 127.4.74.130:40973)
0927 23:00:44.709359 (+ 15899us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:44.709369 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:44.709403 (+ 34us) server_negotiation.cc:407] Connection header received
0927 23:00:44.709528 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:44.709536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:44.709862 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:44.710114 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:00:44.717273 (+ 7159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.718866 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.721678 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.723336 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.723508 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.727098 (+ 3590us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:44.727205 (+ 107us) server_negotiation.cc:299] Negotiation successful
0927 23:00:44.727442 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15669,"thread_start_us":15542,"threads_started":1}
I20260927 23:00:44.726747 17823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.693072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:37023 (local address 127.0.0.1:57140)
0927 23:00:44.707365 (+ 14293us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:44.707387 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:44.707574 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:44.711313 (+ 3739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:44.711319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:44.711346 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:00:44.711762 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.711775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.714146 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.714152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.716984 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.717007 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.725754 (+ 8747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.725762 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.725959 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.725997 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 23:00:44.726096 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 23:00:44.726261 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14136,"thread_start_us":24994,"threads_started":1}
I20260927 23:00:44.729162 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.693849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60522 (local address 127.4.74.129:45151)
0927 23:00:44.704752 (+ 10903us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:44.704761 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:44.718573 (+ 13812us) server_negotiation.cc:407] Connection header received
0927 23:00:44.718697 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:44.718705 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:44.719023 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:44.719288 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:00:44.721266 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.722858 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.725299 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.726761 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.726894 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.727138 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:44.728651 (+ 1513us) server_negotiation.cc:299] Negotiation successful
0927 23:00:44.728835 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10708,"thread_start_us":10531,"threads_started":1}
I20260927 23:00:44.728165 17828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.693099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:45151 (local address 127.0.0.1:60522)
0927 23:00:44.717907 (+ 24808us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:44.717929 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:44.718394 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:44.720486 (+ 2092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:44.720490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:44.720529 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:00:44.721048 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.721063 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.723038 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.723044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.725090 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.725112 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.727292 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.727299 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.727482 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.727531 (+ 49us) client_negotiation.cc:769] Sending connection context
0927 23:00:44.727637 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 23:00:44.727792 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24666,"thread_start_us":10283,"threads_started":1}
I20260927 23:00:44.729923 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.693780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57140 (local address 127.4.74.131:37023)
0927 23:00:44.704188 (+ 10408us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:44.704204 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:44.707737 (+ 3533us) server_negotiation.cc:407] Connection header received
0927 23:00:44.707881 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:44.707891 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:44.708261 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:44.708525 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:00:44.712052 (+ 3527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.713937 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.721066 (+ 7129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.722754 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.722942 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.723254 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:44.729296 (+ 6042us) server_negotiation.cc:299] Negotiation successful
0927 23:00:44.729540 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10116,"thread_start_us":10979,"threads_started":1}
I20260927 23:00:44.734663 17463 tablet_service.cc:1467] Processing CreateTablet for tablet 55e967e521ef4ce18f18310b870bcdb2 (DEFAULT_TABLE table=test-table [id=4d76a22403974d20b1978a53baab408c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:00:44.735049 17741 tablet_service.cc:1467] Processing CreateTablet for tablet 55e967e521ef4ce18f18310b870bcdb2 (DEFAULT_TABLE table=test-table [id=4d76a22403974d20b1978a53baab408c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:00:44.736549 17463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e967e521ef4ce18f18310b870bcdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:44.736956 17741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e967e521ef4ce18f18310b870bcdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:44.753086 17607 tablet_service.cc:1467] Processing CreateTablet for tablet 55e967e521ef4ce18f18310b870bcdb2 (DEFAULT_TABLE table=test-table [id=4d76a22403974d20b1978a53baab408c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:00:44.755162 17607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e967e521ef4ce18f18310b870bcdb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:44.780496 17829 tablet_bootstrap.cc:492] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Bootstrap starting.
I20260927 23:00:44.789213 17830 tablet_bootstrap.cc:492] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb: Bootstrap starting.
I20260927 23:00:44.790246 17829 tablet_bootstrap.cc:654] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:44.791736 17829 log.cc:824] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:44.794122 17830 tablet_bootstrap.cc:654] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:44.795742 17830 log.cc:824] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:44.798131 17832 tablet_bootstrap.cc:492] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: Bootstrap starting.
I20260927 23:00:44.809293 17829 tablet_bootstrap.cc:492] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: No bootstrap required, opened a new log
I20260927 23:00:44.809887 17829 ts_tablet_manager.cc:1397] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.004s
I20260927 23:00:44.820603 17832 tablet_bootstrap.cc:654] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:44.823171 17832 log.cc:824] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:44.823264 17829 raft_consensus.cc:348] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45b8872725a547f98408d6f7d54b4ffe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40973 } } peers { permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 } } peers { permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 } }
I20260927 23:00:44.823886 17830 tablet_bootstrap.cc:492] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb: No bootstrap required, opened a new log
I20260927 23:00:44.824160 17829 raft_consensus.cc:374] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:44.824229 17830 ts_tablet_manager.cc:1397] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb: Time spent bootstrapping tablet: real 0.035s user 0.013s sys 0.019s
I20260927 23:00:44.824455 17829 raft_consensus.cc:729] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a20e944716a44ea8d386d3fb280397d, State: Initialized, Role: FOLLOWER
I20260927 23:00:44.825351 17829 consensus_queue.cc:260] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45b8872725a547f98408d6f7d54b4ffe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40973 } } peers { permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 } } peers { permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 } }
I20260927 23:00:44.833777 17829 ts_tablet_manager.cc:1428] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Time spent starting tablet: real 0.024s user 0.012s sys 0.008s
I20260927 23:00:44.834276 17806 heartbeater.cc:503] Master 127.4.74.190:34461 was elected leader, sending a full tablet report...
I20260927 23:00:44.834306 17832 tablet_bootstrap.cc:492] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: No bootstrap required, opened a new log
I20260927 23:00:44.834719 17832 ts_tablet_manager.cc:1397] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: Time spent bootstrapping tablet: real 0.037s user 0.015s sys 0.000s
I20260927 23:00:44.834870 17672 heartbeater.cc:503] Master 127.4.74.190:34461 was elected leader, sending a full tablet report...
I20260927 23:00:44.843956 17830 raft_consensus.cc:348] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45b8872725a547f98408d6f7d54b4ffe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40973 } } peers { permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 } } peers { permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 } }
I20260927 23:00:44.844565 17830 raft_consensus.cc:374] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:44.844718 17830 raft_consensus.cc:729] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a6ea00a48b4af58712346a02cd45fb, State: Initialized, Role: FOLLOWER
I20260927 23:00:44.845535 17830 consensus_queue.cc:260] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45b8872725a547f98408d6f7d54b4ffe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40973 } } peers { permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 } } peers { permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 } }
I20260927 23:00:44.849148 17832 raft_consensus.cc:348] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45b8872725a547f98408d6f7d54b4ffe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40973 } } peers { permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 } } peers { permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 } }
I20260927 23:00:44.849865 17832 raft_consensus.cc:374] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:44.850060 17832 raft_consensus.cc:729] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b8872725a547f98408d6f7d54b4ffe, State: Initialized, Role: FOLLOWER
I20260927 23:00:44.851092 17832 consensus_queue.cc:260] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45b8872725a547f98408d6f7d54b4ffe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40973 } } peers { permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 } } peers { permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 } }
I20260927 23:00:44.853582 17528 heartbeater.cc:503] Master 127.4.74.190:34461 was elected leader, sending a full tablet report...
I20260927 23:00:44.858819 17830 ts_tablet_manager.cc:1428] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb: Time spent starting tablet: real 0.034s user 0.016s sys 0.007s
I20260927 23:00:44.859567 17832 ts_tablet_manager.cc:1428] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: Time spent starting tablet: real 0.025s user 0.013s sys 0.009s
I20260927 23:00:44.870546 17837 raft_consensus.cc:482] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:00:44.871140 17837 raft_consensus.cc:504] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45b8872725a547f98408d6f7d54b4ffe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40973 } } peers { permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 } } peers { permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 } }
I20260927 23:00:44.884464 17837 leader_election.cc:288] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a20e944716a44ea8d386d3fb280397d (127.4.74.131:37023), 00a6ea00a48b4af58712346a02cd45fb (127.4.74.129:45151)
I20260927 23:00:44.895120 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.877843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:46429 (local address 127.4.74.131:37023)
0927 23:00:44.878120 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:44.878128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:44.887255 (+ 9127us) server_negotiation.cc:407] Connection header received
0927 23:00:44.887595 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:44.887602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:44.887677 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:44.887740 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:00:44.888839 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.889900 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.892723 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.893897 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.894057 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.894133 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:44.894693 (+ 560us) server_negotiation.cc:299] Negotiation successful
0927 23:00:44.894836 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 23:00:44.896000 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.877864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:45651 (local address 127.4.74.129:45151)
0927 23:00:44.878080 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:44.878087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:44.886534 (+ 8447us) server_negotiation.cc:407] Connection header received
0927 23:00:44.887289 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:44.887297 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:44.887418 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:44.887500 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:00:44.888714 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.889869 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.892996 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.894339 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.894506 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.894599 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:44.895228 (+ 629us) server_negotiation.cc:299] Negotiation successful
0927 23:00:44.895416 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 23:00:44.896111 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.877144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:37023 (local address 127.4.74.130:46429)
0927 23:00:44.887094 (+ 9950us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:44.887124 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:44.887392 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:44.887970 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:44.887975 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:44.888059 (+ 84us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:00:44.888651 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.888669 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.890111 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.890117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.892533 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.892549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.894328 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.894334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.894503 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.894536 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 23:00:44.895500 (+ 964us) client_negotiation.cc:240] Negotiation successful
0927 23:00:44.895674 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9775,"mutex_wait_us":3,"spinlock_wait_cycles":1280,"thread_start_us":8842,"threads_started":1}
I20260927 23:00:44.897258 17839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.876343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:45151 (local address 127.4.74.130:45651)
0927 23:00:44.886322 (+ 9979us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:44.886357 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:44.886636 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:44.887796 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:44.887802 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:44.887833 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:00:44.888499 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.888520 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.890356 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.890366 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.892770 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:44.892787 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.894823 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.894830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:44.895014 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.895050 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 23:00:44.896648 (+ 1598us) client_negotiation.cc:240] Negotiation successful
0927 23:00:44.896835 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9746,"spinlock_wait_cycles":11136,"thread_start_us":10439,"threads_started":1}
I20260927 23:00:44.897204 17761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e967e521ef4ce18f18310b870bcdb2" candidate_uuid: "45b8872725a547f98408d6f7d54b4ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a20e944716a44ea8d386d3fb280397d" is_pre_election: true
I20260927 23:00:44.897774 17761 raft_consensus.cc:2454] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45b8872725a547f98408d6f7d54b4ffe in term 0.
I20260927 23:00:44.898622 17560 leader_election.cc:302] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b8872725a547f98408d6f7d54b4ffe, 8a20e944716a44ea8d386d3fb280397d; no voters:
I20260927 23:00:44.898748 17483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e967e521ef4ce18f18310b870bcdb2" candidate_uuid: "45b8872725a547f98408d6f7d54b4ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a6ea00a48b4af58712346a02cd45fb" is_pre_election: true
I20260927 23:00:44.899092 17837 raft_consensus.cc:2781] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:00:44.899226 17837 raft_consensus.cc:482] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:00:44.899309 17837 raft_consensus.cc:3037] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:44.899430 17483 raft_consensus.cc:2454] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45b8872725a547f98408d6f7d54b4ffe in term 0.
I20260927 23:00:44.903014 17837 raft_consensus.cc:504] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45b8872725a547f98408d6f7d54b4ffe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40973 } } peers { permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 } } peers { permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 } }
I20260927 23:00:44.904474 17761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e967e521ef4ce18f18310b870bcdb2" candidate_uuid: "45b8872725a547f98408d6f7d54b4ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a20e944716a44ea8d386d3fb280397d"
I20260927 23:00:44.904784 17761 raft_consensus.cc:3037] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:44.904893 17483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e967e521ef4ce18f18310b870bcdb2" candidate_uuid: "45b8872725a547f98408d6f7d54b4ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a6ea00a48b4af58712346a02cd45fb"
I20260927 23:00:44.905174 17483 raft_consensus.cc:3037] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:44.908677 17483 raft_consensus.cc:2454] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45b8872725a547f98408d6f7d54b4ffe in term 1.
I20260927 23:00:44.908900 17761 raft_consensus.cc:2454] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45b8872725a547f98408d6f7d54b4ffe in term 1.
I20260927 23:00:44.909390 17558 leader_election.cc:302] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a6ea00a48b4af58712346a02cd45fb, 45b8872725a547f98408d6f7d54b4ffe; no voters:
I20260927 23:00:44.914019 17837 leader_election.cc:288] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [CANDIDATE]: Term 1 election: Requested vote from peers 8a20e944716a44ea8d386d3fb280397d (127.4.74.131:37023), 00a6ea00a48b4af58712346a02cd45fb (127.4.74.129:45151)
I20260927 23:00:44.914628 17837 raft_consensus.cc:2781] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:00:44.915082 17837 raft_consensus.cc:686] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 1 LEADER]: Becoming Leader. State: Replica: 45b8872725a547f98408d6f7d54b4ffe, State: Running, Role: LEADER
I20260927 23:00:44.915959 17837 consensus_queue.cc:237] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45b8872725a547f98408d6f7d54b4ffe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40973 } } peers { permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 } } peers { permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 } }
W20260927 23:00:44.918509 17529 tablet.cc:2369] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:00:44.935282 17331 catalog_manager.cc:5495] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe reported cstate change: term changed from 0 to 1, leader changed from <none> to 45b8872725a547f98408d6f7d54b4ffe (127.4.74.130). New cstate: current_term: 1 leader_uuid: "45b8872725a547f98408d6f7d54b4ffe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45b8872725a547f98408d6f7d54b4ffe" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 } health_report { overall_health: UNKNOWN } } }
I20260927 23:00:44.992065 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:44.986210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40100 (local address 127.4.74.130:40973)
0927 23:00:44.986389 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:44.986393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:44.986410 (+ 17us) server_negotiation.cc:407] Connection header received
0927 23:00:44.986504 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:44.986508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:44.986596 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:44.986773 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:00:44.987180 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.988056 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.989640 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:44.990076 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:44.990209 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:44.990774 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:00:44.991339 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:00:44.991509 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:44.991604 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 23:00:44.991714 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 23:00:45.006039 17483 raft_consensus.cc:1261] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb [term 1 FOLLOWER]: Refusing update from remote peer 45b8872725a547f98408d6f7d54b4ffe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:00:45.006251 17761 raft_consensus.cc:1261] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [term 1 FOLLOWER]: Refusing update from remote peer 45b8872725a547f98408d6f7d54b4ffe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:00:45.007467 17837 consensus_queue.cc:1035] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a20e944716a44ea8d386d3fb280397d" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:00:45.007958 17840 consensus_queue.cc:1035] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [LEADER]: Connected to new peer: Peer: permanent_uuid: "00a6ea00a48b4af58712346a02cd45fb" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 45151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:00:45.060767 17807 tablet.cc:2369] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:00:45.064215 17331 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:00:45.064597 17331 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:00:45.071709 17331 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2aaf41e46ff4a449e03fff1d9f99d24: Sending DeleteTablet for 3 replicas of tablet 55e967e521ef4ce18f18310b870bcdb2
I20260927 23:00:45.073246 17845 mvcc.cc:205] Tried to move back new op lower bound from 7334092984330199040 to 7334092983988142080. Current Snapshot: MvccSnapshot[applied={T|T < 7334092984330199040}]
I20260927 23:00:45.074105 17607 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e967e521ef4ce18f18310b870bcdb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:00:45 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:40096
I20260927 23:00:45.075285 17741 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e967e521ef4ce18f18310b870bcdb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:00:45 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:57140
I20260927 23:00:45.080173 17853 tablet_replica.cc:330] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: stopping tablet replica
I20260927 23:00:45.080663 17463 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e967e521ef4ce18f18310b870bcdb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:00:45 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:60522
I20260927 23:00:45.080842 17853 raft_consensus.cc:2229] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:00:45.081635 17853 raft_consensus.cc:2258] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:00:45.082556 17854 tablet_replica.cc:330] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: stopping tablet replica
I20260927 23:00:45.083050 17854 raft_consensus.cc:2229] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:00:45.083566 17854 pending_rounds.cc:62] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Trying to abort 2 pending ops.
I20260927 23:00:45.083662 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 17368
I20260927 23:00:45.083716 17854 pending_rounds.cc:68] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7334092984330199040 op_type: WRITE_OP write_request { tablet_id: "55e967e521ef4ce18f18310b870bcdb2" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\310\302\346\325\006\"8\n\ntest-admin\022*\n 4d76a22403974d20b1978a53baab408c\020\001\030\001 \001(\001" signature: "<redacted>""\t\315B\221p\366\277\202\322\032\341\257\311j\022?8U\350%\360]\373\274\3557{\207\345\360\030\035CV&u\016\340\330\204&\251h\216\203\354\251\204\301\200k^\332-\325d\201\262\311\\\363c\340\355" signing_key_seq_num: 0 } } request_id { client_id: "db113592a53a47fbad333ae0764efed8" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260927 23:00:45.085467 17854 pending_rounds.cc:68] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334092983988142080 op_type: NO_OP noop_request { }
I20260927 23:00:45.085655 17854 raft_consensus.cc:2866] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:00:45.085831 17854 raft_consensus.cc:2258] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:00:45.098428 17855 tablet_replica.cc:330] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb: stopping tablet replica
W20260927 23:00:45.100481 17850 consensus_queue.cc:1159] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8a20e944716a44ea8d386d3fb280397d" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260927 23:00:45.102862 17855 raft_consensus.cc:2229] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:00:45.104164 17855 raft_consensus.cc:2258] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:00:45.107077 17853 ts_tablet_manager.cc:1905] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:00:45.110339 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.086519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.4.74.129:45151)
0927 23:00:45.086714 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:45.086720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:45.086742 (+ 22us) server_negotiation.cc:407] Connection header received
0927 23:00:45.086812 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:45.086818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:45.086903 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:45.087024 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:45.090505 (+ 3481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:45.091694 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:45.097744 (+ 6050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:45.098524 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:45.098704 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:45.102395 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:45.102413 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:45.102433 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:45.102484 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:45.106617 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:45.107817 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:45.107827 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:45.107831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:45.108031 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:45.108492 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:45.108497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:45.108501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:45.108946 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:45.109291 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:45.109616 (+ 325us) server_negotiation.cc:299] Negotiation successful
0927 23:00:45.109880 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 23:00:45.110380 17855 ts_tablet_manager.cc:1905] T 55e967e521ef4ce18f18310b870bcdb2 P 00a6ea00a48b4af58712346a02cd45fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:00:45.111027 17523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:45.122020 17854 ts_tablet_manager.cc:1905] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:00:45.126557 17853 ts_tablet_manager.cc:1918] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:00:45.126803 17853 log.cc:1196] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/55e967e521ef4ce18f18310b870bcdb2
I20260927 23:00:45.127398 17853 ts_tablet_manager.cc:1939] T 55e967e521ef4ce18f18310b870bcdb2 P 45b8872725a547f98408d6f7d54b4ffe: Deleting consensus metadata
I20260927 23:00:45.129920 17318 catalog_manager.cc:4841] TS 45b8872725a547f98408d6f7d54b4ffe (127.4.74.130:40973): tablet 55e967e521ef4ce18f18310b870bcdb2 (table test-table [id=4d76a22403974d20b1978a53baab408c]) successfully deleted
I20260927 23:00:45.133189 17854 ts_tablet_manager.cc:1918] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:00:45.133481 17854 log.cc:1196] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/55e967e521ef4ce18f18310b870bcdb2
I20260927 23:00:45.134281 17854 ts_tablet_manager.cc:1939] T 55e967e521ef4ce18f18310b870bcdb2 P 8a20e944716a44ea8d386d3fb280397d: Deleting consensus metadata
I20260927 23:00:45.136600 17317 catalog_manager.cc:4841] TS 8a20e944716a44ea8d386d3fb280397d (127.4.74.131:37023): tablet 55e967e521ef4ce18f18310b870bcdb2 (table test-table [id=4d76a22403974d20b1978a53baab408c]) successfully deleted
I20260927 23:00:45.229056 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 17368
I20260927 23:00:45.249294 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 17532
W20260927 23:00:45.249179 17315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.74.129:45151 (error 108)
I20260927 23:00:45.274309 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.250252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46044 (local address 127.4.74.130:40973)
0927 23:00:45.250474 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:45.250480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:45.250608 (+ 128us) server_negotiation.cc:407] Connection header received
0927 23:00:45.250865 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:45.250874 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:45.250998 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:45.251109 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:45.252602 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:45.254500 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:45.255883 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:45.256814 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:45.256961 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:45.261700 (+ 4739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:45.261728 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:45.261762 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:45.261864 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:45.267763 (+ 5899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:45.269285 (+ 1522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:45.269302 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:45.269309 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:45.269572 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:45.270210 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:45.270219 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:45.270225 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:45.270924 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:45.271385 (+ 461us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:45.273195 (+ 1810us) server_negotiation.cc:299] Negotiation successful
0927 23:00:45.273631 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 23:00:45.276222 17667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:45.277482 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.266605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:45151 (local address 127.0.0.1:60738)
0927 23:00:45.277046 (+ 10441us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:45.277241 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:45151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10265,"thread_start_us":10064,"threads_started":1}
W20260927 23:00:45.277897 17315 catalog_manager.cc:4584] TS 00a6ea00a48b4af58712346a02cd45fb (127.4.74.129:45151): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55e967e521ef4ce18f18310b870bcdb2: Network error: Client connection negotiation failed: client connection to 127.4.74.129:45151: connect: Connection refused (error 111)
I20260927 23:00:45.332125 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.331374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:45151 (local address 127.0.0.1:60752)
0927 23:00:45.331757 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:45.331932 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:45151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260927 23:00:45.333705 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.333152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:45151 (local address 127.0.0.1:60756)
0927 23:00:45.333349 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:45.333483 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:45151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260927 23:00:45.394740 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.394003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:45151 (local address 127.0.0.1:60772)
0927 23:00:45.394327 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:45.394486 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:45151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260927 23:00:45.396196 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.395769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:45151 (local address 127.0.0.1:60778)
0927 23:00:45.395928 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:45.396022 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:45151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 23:00:45.407424 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 17532
I20260927 23:00:45.428930 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 17676
I20260927 23:00:45.445698 17865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.429785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42372 (local address 127.4.74.131:37023)
0927 23:00:45.430341 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:45.430345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:45.430367 (+ 22us) server_negotiation.cc:407] Connection header received
0927 23:00:45.430436 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:45.430440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:45.430509 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:45.430631 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:45.432377 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:45.433164 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:45.434684 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:45.435114 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:45.435227 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:45.438628 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:45.438643 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:45.438659 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:45.438710 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:45.441893 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:45.442884 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:45.442892 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:45.442895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:45.443132 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:45.443970 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:45.443975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:45.443979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:45.444591 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:45.444824 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:45.445214 (+ 390us) server_negotiation.cc:299] Negotiation successful
0927 23:00:45.445405 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":335,"threads_started":1}
I20260927 23:00:45.446341 17801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:45.477324 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.476867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:45151 (local address 127.0.0.1:60794)
0927 23:00:45.477068 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:45.477168 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:45151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260927 23:00:45.478382 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.477979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:45151 (local address 127.0.0.1:60800)
0927 23:00:45.478166 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:45.478242 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:45151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260927 23:00:45.548586 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 17676
I20260927 23:00:45.567953 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 17298
I20260927 23:00:45.569347 17361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:45.686327 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:45.685254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:45151 (local address 127.0.0.1:60812)
0927 23:00:45.685802 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:45.686008 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:45151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349}
I20260927 23:00:45.686566 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 17298
I20260927 23:00:45.713596 4394 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 23:00:46.060529 4394 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 23:00:46.060755 4394 mini_ranger.cc:66] Stopping Ranger...
I20260927 23:00:46.407899 4394 mini_ranger.cc:68] Stopped Ranger
2026-09-27 23:00:46.408 UTC [16349] LOG: received smart shutdown request
2026-09-27 23:00:46.412 UTC [16349] LOG: background worker "logical replication launcher" (PID 16358) exited with exit code 1
2026-09-27 23:00:46.412 UTC [16353] LOG: shutting down
2026-09-27 23:00:46.608 UTC [16349] LOG: database system is shut down
2026-09-27T23:00:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (59993 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:00:46 dist-test-slave-5pd3 krb5kdc[17876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:00:46 dist-test-slave-5pd3 krb5kdc[17876](info): set up 2 sockets
Sep 27 23:00:46 dist-test-slave-5pd3 krb5kdc[17876](info): commencing operation
krb5kdc: starting...
W20260927 23:00:47.936266 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.006s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:00:48 dist-test-slave-5pd3 krb5kdc[17876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:00:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:00:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:00:48.214696 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36721
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36721,127.4.74.189:42321,127.4.74.188:41723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:00:48.557323 17889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:48.557693 17889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:48.557857 17889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:48.565461 17889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:00:48.565593 17889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:48.565680 17889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:48.565783 17889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:00:48.565864 17889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:00:48.576421 17889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36721,127.4.74.189:42321,127.4.74.188:41723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:48.577883 17889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:48.579211 17889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:48.592445 17894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:48.592545 17895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:48.592516 17897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:48.593571 17889 server_base.cc:1034] running on GCE node
I20260927 23:00:48.594472 17889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:48.596138 17889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:48.597383 17889 hybrid_clock.cc:648] HybridClock initialized: now 1790550048597324 us; error 56 us; skew 500 ppm
I20260927 23:00:48.597816 17889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:48 dist-test-slave-5pd3 krb5kdc[17876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550048, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:48.612550 17889 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:00:48.614521 17889 webserver.cc:460] Webserver started at http://127.4.74.190:42615/ using document root <none> and password file <none>
I20260927 23:00:48.615289 17889 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:48.615429 17889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:48.615826 17889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:00:48.618984 17889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "3f4dca9310ce404ead56836f7f6fd398"
format_stamp: "Formatted at 2026-09-27 23:00:48 on dist-test-slave-5pd3"
server_key: "b5984b697d70cd7a0f83c6d32ee54870"
server_key_iv: "5e9d752dc26b886785a02086dfab2f03"
server_key_version: "encryptionkey@0"
I20260927 23:00:48.619974 17889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "3f4dca9310ce404ead56836f7f6fd398"
format_stamp: "Formatted at 2026-09-27 23:00:48 on dist-test-slave-5pd3"
server_key: "b5984b697d70cd7a0f83c6d32ee54870"
server_key_iv: "5e9d752dc26b886785a02086dfab2f03"
server_key_version: "encryptionkey@0"
I20260927 23:00:48.625861 17889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260927 23:00:48.629931 17904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:48.631937 17889 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:00:48.632161 17889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "3f4dca9310ce404ead56836f7f6fd398"
format_stamp: "Formatted at 2026-09-27 23:00:48 on dist-test-slave-5pd3"
server_key: "b5984b697d70cd7a0f83c6d32ee54870"
server_key_iv: "5e9d752dc26b886785a02086dfab2f03"
server_key_version: "encryptionkey@0"
I20260927 23:00:48.632405 17889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:48.651695 17889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:48.656442 17889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:48.656764 17889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:48.678570 17889 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:36721
I20260927 23:00:48.678682 17955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:36721 every 8 connection(s)
I20260927 23:00:48.680138 17889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:00:48.684324 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 17889
I20260927 23:00:48.684623 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:00:48.685499 17956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:48.685567 4394 external_mini_cluster.cc:1428] Setting key 9fb26143575ae75025a9ecf904cf625a
I20260927 23:00:48.697994 17956 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.74.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.189@KRBTEST.COM" created.
Sep 27 23:00:48 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550048, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:48.728145 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:48.702667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36721 (local address 127.0.0.1:42990)
0927 23:00:48.704404 (+ 1737us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:48.704451 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:48.706717 (+ 2266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:48.708371 (+ 1654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:48.708378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:48.709235 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:48.709720 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:48.709741 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:48.712480 (+ 2739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:48.712485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:48.714106 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:48.714141 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:48.715667 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:48.715678 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:48.715956 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:48.716898 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:48.716941 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:48.720352 (+ 3411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:48.723864 (+ 3512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:48.723875 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:48.723888 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:48.724293 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:48.724716 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:48.724720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:48.724724 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:48.724870 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:48.725655 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:48.725663 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:48.725871 (+ 208us) client_negotiation.cc:769] Sending connection context
0927 23:00:48.726134 (+ 263us) client_negotiation.cc:240] Negotiation successful
0927 23:00:48.726731 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1451,"mutex_wait_us":151,"spinlock_wait_cycles":17792,"thread_start_us":1142,"threads_started":1}
I20260927 23:00:48.728170 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:48.702664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.4.74.190:36721)
0927 23:00:48.704026 (+ 1362us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:48.704047 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:48.704616 (+ 569us) server_negotiation.cc:407] Connection header received
0927 23:00:48.707346 (+ 2730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:48.707359 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:48.707946 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:48.708280 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:48.710043 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:48.712239 (+ 2196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:48.714370 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:48.715261 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:48.715428 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:48.720616 (+ 5188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:48.720624 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:48.720638 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:48.720671 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:48.723729 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:48.724428 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:48.724443 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:48.724446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:48.724589 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:48.724972 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:48.724980 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:48.724983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:48.725508 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:48.725724 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:48.726243 (+ 519us) server_negotiation.cc:299] Negotiation successful
0927 23:00:48.726619 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1036,"thread_start_us":696,"threads_started":1}
I20260927 23:00:48.732435 17956 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:00:48.734097 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:48.733503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59202)
0927 23:00:48.733770 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:48.733912 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260927 23:00:48.734639 17905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
I20260927 23:00:48.736737 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:48.736337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59204)
0927 23:00:48.736495 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:48.736581 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260927 23:00:48.736989 17956 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:42321: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab.
Entry for principal kudu/127.4.74.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab.
I20260927 23:00:48.752564 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.189:42321
--webserver_interface=127.4.74.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab
--principal=kudu/127.4.74.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36721,127.4.74.189:42321,127.4.74.188:41723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:00:48.786311 17956 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } attempt: 1
I20260927 23:00:48.787441 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:48.786992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59210)
0927 23:00:48.787211 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:48.787318 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260927 23:00:48.788476 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:48.788115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59226)
0927 23:00:48.788252 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:48.788324 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 23:00:48.788659 17956 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:42321: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
I20260927 23:00:48.856985 17956 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } attempt: 2
I20260927 23:00:48.858522 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:48.857746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59228)
0927 23:00:48.858083 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:48.858262 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260927 23:00:48.860172 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:48.859539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59238)
0927 23:00:48.859817 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:48.859941 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260927 23:00:48.860476 17956 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:42321: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
I20260927 23:00:48.951862 17956 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } attempt: 3
I20260927 23:00:48.953563 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:48.952898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59248)
0927 23:00:48.953201 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:48.953350 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260927 23:00:48.954918 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:48.954424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59252)
0927 23:00:48.954601 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:48.954714 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260927 23:00:48.955176 17956 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:42321: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
I20260927 23:00:49.098563 17956 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } attempt: 4
I20260927 23:00:49.100333 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.099410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59260)
0927 23:00:49.099899 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.100073 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234}
I20260927 23:00:49.101853 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.101315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59264)
0927 23:00:49.101512 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.101649 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260927 23:00:49.102134 17956 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:42321: Network error: Client connection negotiation failed: client connection to 127.4.74.189:42321: connect: Connection refused (error 111)
W20260927 23:00:49.180370 17963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:49.180696 17963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:49.180853 17963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:49.188405 17963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:00:49.188526 17963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:49.188608 17963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:49.188686 17963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:00:49.188761 17963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:00:49.199232 17963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36721,127.4.74.189:42321,127.4.74.188:41723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab
--principal=kudu/127.4.74.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.189:42321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.74.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:49.200732 17963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:49.202216 17963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:49.216876 17971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:49.216954 17968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:49.217015 17969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:49.218235 17963 server_base.cc:1034] running on GCE node
I20260927 23:00:49.219270 17963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:49.221063 17963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:49.222344 17963 hybrid_clock.cc:648] HybridClock initialized: now 1790550049222268 us; error 80 us; skew 500 ppm
I20260927 23:00:49.222771 17963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550049, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:49.226648 17963 init.cc:377] Logged in from keytab as kudu/127.4.74.189@KRBTEST.COM (short username kudu)
I20260927 23:00:49.228657 17963 webserver.cc:460] Webserver started at http://127.4.74.189:43817/ using document root <none> and password file <none>
I20260927 23:00:49.229477 17963 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:49.229640 17963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:49.229997 17963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:00:49.233457 17963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/data/instance:
uuid: "4a9dd2404eac410b93b33a610ce7ebcd"
format_stamp: "Formatted at 2026-09-27 23:00:49 on dist-test-slave-5pd3"
server_key: "723cdc0ecdd2140558f60e6f50a73b2d"
server_key_iv: "47b96c20ce359237cd70de5a6a6a49e4"
server_key_version: "encryptionkey@0"
I20260927 23:00:49.234367 17963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/wal/instance:
uuid: "4a9dd2404eac410b93b33a610ce7ebcd"
format_stamp: "Formatted at 2026-09-27 23:00:49 on dist-test-slave-5pd3"
server_key: "723cdc0ecdd2140558f60e6f50a73b2d"
server_key_iv: "47b96c20ce359237cd70de5a6a6a49e4"
server_key_version: "encryptionkey@0"
I20260927 23:00:49.241258 17963 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260927 23:00:49.245684 17978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:49.247601 17963 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 23:00:49.247839 17963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/wal
uuid: "4a9dd2404eac410b93b33a610ce7ebcd"
format_stamp: "Formatted at 2026-09-27 23:00:49 on dist-test-slave-5pd3"
server_key: "723cdc0ecdd2140558f60e6f50a73b2d"
server_key_iv: "47b96c20ce359237cd70de5a6a6a49e4"
server_key_version: "encryptionkey@0"
I20260927 23:00:49.248070 17963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:49.277025 17963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:49.281973 17963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:49.282308 17963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:49.305260 17963 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.189:42321
I20260927 23:00:49.305328 18029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.189:42321 every 8 connection(s)
I20260927 23:00:49.306836 17963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
I20260927 23:00:49.311919 18030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:49.314533 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 17963
I20260927 23:00:49.314769 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-1/wal/instance
I20260927 23:00:49.315402 4394 external_mini_cluster.cc:1428] Setting key 5816f624e7f83e2f72dc24457a8d1107
I20260927 23:00:49.323408 18030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.74.188@KRBTEST.COM; defaulting to no policy
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550049, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.188@KRBTEST.COM" created.
I20260927 23:00:49.352128 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.328434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43004 (local address 127.4.74.190:36721)
0927 23:00:49.329074 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.329079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.330484 (+ 1405us) server_negotiation.cc:407] Connection header received
0927 23:00:49.332984 (+ 2500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.332991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.333116 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.333258 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:49.335314 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.336315 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.338332 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.338771 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.338932 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.343811 (+ 4879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:49.343820 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:49.343824 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:49.343889 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:49.347248 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.348833 (+ 1585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.348839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.348842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.348989 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.349730 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.349735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.349737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.350089 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:49.350231 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:49.351454 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0927 23:00:49.351718 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":370,"threads_started":1}
I20260927 23:00:49.354022 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.329209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36721 (local address 127.0.0.1:43004)
0927 23:00:49.330256 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.330317 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:49.332604 (+ 2287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:49.333601 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:49.333613 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:49.334317 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:49.335079 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.335100 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.336500 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.336506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.338163 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.338178 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.339045 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.339052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.339307 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.340246 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:49.340272 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:49.343395 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:49.347533 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.347615 (+ 82us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.347645 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.348617 (+ 972us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.349157 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.349166 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.349172 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.349507 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.350259 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:49.350273 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:49.350973 (+ 700us) client_negotiation.cc:769] Sending connection context
0927 23:00:49.351354 (+ 381us) client_negotiation.cc:240] Negotiation successful
0927 23:00:49.352124 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":431,"threads_started":1}
I20260927 23:00:49.356503 18030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550049, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
I20260927 23:00:49.377017 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.357552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59276)
0927 23:00:49.359229 (+ 1677us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.359250 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:49.359395 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:49.360165 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:49.360170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:49.360633 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:49.360828 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.360837 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.363082 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.363088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.364150 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.364165 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.365712 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.365718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.365924 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.366776 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:49.366799 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:49.369256 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:49.373610 (+ 4354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.373616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.373619 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.374234 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.374761 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.374765 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.374767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.374988 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.375933 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:49.375937 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:49.376104 (+ 167us) client_negotiation.cc:769] Sending connection context
0927 23:00:49.376247 (+ 143us) client_negotiation.cc:240] Negotiation successful
0927 23:00:49.376632 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260927 23:00:49.377211 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.358162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59276 (local address 127.4.74.189:42321)
0927 23:00:49.359413 (+ 1251us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.359425 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.359465 (+ 40us) server_negotiation.cc:407] Connection header received
0927 23:00:49.359634 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.359646 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.359950 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.360142 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:49.361017 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.362885 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.364325 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.365431 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.365609 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.369562 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:49.369582 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:49.369615 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:49.369674 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:49.373414 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.374406 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.374414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.374416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.374630 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.375183 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.375192 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.375197 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.375740 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:49.376023 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:49.376438 (+ 415us) server_negotiation.cc:299] Negotiation successful
0927 23:00:49.376760 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":1000,"thread_start_us":653,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab.
Entry for principal kudu/127.4.74.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab.
I20260927 23:00:49.380007 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.188:41723
--webserver_interface=127.4.74.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab
--principal=kudu/127.4.74.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36721,127.4.74.189:42321,127.4.74.188:41723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:00:49.380515 18030 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:00:49.382072 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.381286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54828)
0927 23:00:49.381577 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.381811 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260927 23:00:49.382411 17982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
I20260927 23:00:49.384573 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.384038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54838)
0927 23:00:49.384251 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.384386 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260927 23:00:49.384902 18030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:41723: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
I20260927 23:00:49.401456 17956 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } attempt: 5
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550048, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
I20260927 23:00:49.419780 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.402510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59292)
0927 23:00:49.402839 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.402869 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:49.403067 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:49.403804 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:49.403809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:49.404530 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:49.404767 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.404780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.406572 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.406576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.407643 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.407656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.408831 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.408840 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.409095 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.410002 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:49.410020 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:49.412611 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:49.416943 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.416948 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.416951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.417411 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.417859 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.417862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.417864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.418004 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.418686 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:49.418690 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:49.418862 (+ 172us) client_negotiation.cc:769] Sending connection context
0927 23:00:49.419026 (+ 164us) client_negotiation.cc:240] Negotiation successful
0927 23:00:49.419411 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260927 23:00:49.419864 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.402821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.4.74.189:42321)
0927 23:00:49.403152 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.403163 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.403197 (+ 34us) server_negotiation.cc:407] Connection header received
0927 23:00:49.403304 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.403313 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.403426 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.403706 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:49.404983 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.406385 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.407806 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.408611 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.408764 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.412867 (+ 4103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:49.412875 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:49.412881 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:49.412931 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:49.416750 (+ 3819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.417548 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.417554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.417559 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.417730 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.418126 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.418132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.418136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.418543 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:49.418710 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:49.419211 (+ 501us) server_negotiation.cc:299] Negotiation successful
0927 23:00:49.419461 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260927 23:00:49.421305 17956 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:00:49.422639 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.422158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54840)
0927 23:00:49.422380 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.422475 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260927 23:00:49.424079 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.423424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54850)
0927 23:00:49.423866 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.423965 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260927 23:00:49.424357 17956 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:41723: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
I20260927 23:00:49.434361 18030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } attempt: 1
I20260927 23:00:49.436002 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.435350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54856)
0927 23:00:49.435639 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.435806 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260927 23:00:49.437355 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.436912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54858)
0927 23:00:49.437075 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.437160 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260927 23:00:49.437717 18030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:41723: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
I20260927 23:00:49.475680 17956 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } attempt: 1
I20260927 23:00:49.477308 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.476732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54870)
0927 23:00:49.476977 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.477102 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260927 23:00:49.478425 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.478051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54876)
0927 23:00:49.478210 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.478307 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260927 23:00:49.478632 17956 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:41723: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
I20260927 23:00:49.505986 18030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } attempt: 2
I20260927 23:00:49.507690 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.506966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54882)
0927 23:00:49.507244 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.507398 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260927 23:00:49.508908 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.508523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54890)
0927 23:00:49.508684 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.508772 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260927 23:00:49.509140 18030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:41723: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
I20260927 23:00:49.546916 17956 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } attempt: 2
I20260927 23:00:49.548445 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.547868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54894)
0927 23:00:49.548153 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.548252 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260927 23:00:49.549482 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.549132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54898)
0927 23:00:49.549278 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.549357 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 23:00:49.549696 17956 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:41723: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
I20260927 23:00:49.600407 18030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } attempt: 3
I20260927 23:00:49.601940 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.601316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54914)
0927 23:00:49.601607 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.601731 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260927 23:00:49.603322 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.602834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54920)
0927 23:00:49.603083 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.603176 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260927 23:00:49.603619 18030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:41723: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
I20260927 23:00:49.655964 17956 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } attempt: 3
I20260927 23:00:49.657406 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.656853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54924)
0927 23:00:49.657096 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.657205 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260927 23:00:49.658553 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.658142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54938)
0927 23:00:49.658339 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.658426 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260927 23:00:49.658782 17956 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:41723: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
I20260927 23:00:49.746927 18030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } attempt: 4
I20260927 23:00:49.748448 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.747846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54948)
0927 23:00:49.748104 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.748229 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260927 23:00:49.749616 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.749248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54952)
0927 23:00:49.749417 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.749498 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260927 23:00:49.749848 18030 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:41723: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
W20260927 23:00:49.754261 18038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:49.754612 18038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:49.754783 18038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:49.762354 18038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:00:49.762521 18038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:49.762655 18038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:49.762758 18038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:00:49.762843 18038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:00:49.773218 18038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36721,127.4.74.189:42321,127.4.74.188:41723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab
--principal=kudu/127.4.74.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.188:41723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.74.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:49.774658 18038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:49.776046 18038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:49.789924 18046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:49.790026 18043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:49.790026 18044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:49.791136 18038 server_base.cc:1034] running on GCE node
I20260927 23:00:49.792191 18038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:49.793804 18038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:49.795130 18038 hybrid_clock.cc:648] HybridClock initialized: now 1790550049795079 us; error 53 us; skew 500 ppm
I20260927 23:00:49.795614 18038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550049, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:49.799160 18038 init.cc:377] Logged in from keytab as kudu/127.4.74.188@KRBTEST.COM (short username kudu)
I20260927 23:00:49.800995 18038 webserver.cc:460] Webserver started at http://127.4.74.188:35375/ using document root <none> and password file <none>
I20260927 23:00:49.801743 18038 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:49.801891 18038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:49.802248 18038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:00:49.805480 18038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/data/instance:
uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec"
format_stamp: "Formatted at 2026-09-27 23:00:49 on dist-test-slave-5pd3"
server_key: "33642f1a72fead7132f85820eb6e6e25"
server_key_iv: "0f485f6140917c90084a51dc66e1dc2f"
server_key_version: "encryptionkey@0"
I20260927 23:00:49.806326 18038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/wal/instance:
uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec"
format_stamp: "Formatted at 2026-09-27 23:00:49 on dist-test-slave-5pd3"
server_key: "33642f1a72fead7132f85820eb6e6e25"
server_key_iv: "0f485f6140917c90084a51dc66e1dc2f"
server_key_version: "encryptionkey@0"
I20260927 23:00:49.812431 18038 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260927 23:00:49.816773 18053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:49.818502 18038 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:00:49.818781 18038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/wal
uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec"
format_stamp: "Formatted at 2026-09-27 23:00:49 on dist-test-slave-5pd3"
server_key: "33642f1a72fead7132f85820eb6e6e25"
server_key_iv: "0f485f6140917c90084a51dc66e1dc2f"
server_key_version: "encryptionkey@0"
I20260927 23:00:49.819018 18038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:49.836024 17956 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } attempt: 4
I20260927 23:00:49.837586 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.837002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54968)
0927 23:00:49.837240 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.837357 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260927 23:00:49.838696 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.838315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:54984)
0927 23:00:49.838468 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.838561 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260927 23:00:49.838913 17956 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:41723: Network error: Client connection negotiation failed: client connection to 127.4.74.188:41723: connect: Connection refused (error 111)
I20260927 23:00:49.865329 18038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:49.870260 18038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:49.870635 18038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:49.892458 18038 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.188:41723
I20260927 23:00:49.892596 18104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.188:41723 every 8 connection(s)
I20260927 23:00:49.894017 18038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
I20260927 23:00:49.899096 18105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:49.900661 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 18038
I20260927 23:00:49.900887 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790549821755302-4394-0/minicluster-data/master-2/wal/instance
I20260927 23:00:49.901575 4394 external_mini_cluster.cc:1428] Setting key 194e053058d4875b18d2720ac144440f
I20260927 23:00:49.911654 18105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550049, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:49.932581 18108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.904186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.4.74.188:41723)
0927 23:00:49.905292 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.905305 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.905361 (+ 56us) server_negotiation.cc:407] Connection header received
0927 23:00:49.907253 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.907280 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.907858 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.908250 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:49.910688 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.912184 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.914540 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.915776 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.916019 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.921673 (+ 5654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:49.921686 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:49.921701 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:49.921776 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:49.925932 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.927008 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.927113 (+ 105us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.927133 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.927495 (+ 362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.928133 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.928141 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.928146 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.929174 (+ 1028us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:49.929556 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:49.930254 (+ 698us) server_negotiation.cc:299] Negotiation successful
0927 23:00:49.930867 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":480,"threads_started":1}
I20260927 23:00:49.934975 18109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.915689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43020 (local address 127.4.74.190:36721)
0927 23:00:49.916305 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.916311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.916917 (+ 606us) server_negotiation.cc:407] Connection header received
0927 23:00:49.917259 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.917265 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.917362 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.917503 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:49.918901 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.920010 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.921794 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.922676 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.922814 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.927314 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:49.927323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:49.927328 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:49.927395 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:49.931357 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.932456 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.932463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.932466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.932659 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.933234 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.933242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.933246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.933647 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:49.933814 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:49.934386 (+ 572us) server_negotiation.cc:299] Negotiation successful
0927 23:00:49.934648 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":307,"threads_started":1}
I20260927 23:00:49.935191 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.915619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36721 (local address 127.0.0.1:43020)
0927 23:00:49.916745 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.916798 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:49.917039 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:49.917578 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:49.917585 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:49.918347 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:49.918703 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.918714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.920238 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.920244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.921553 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.921568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.922908 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.922919 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.923205 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.924013 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:49.924047 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:49.927002 (+ 2955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:49.931643 (+ 4641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.931659 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.931666 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.932302 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.932829 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.932836 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.932841 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.933081 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.933810 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:49.933818 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:49.934030 (+ 212us) client_negotiation.cc:769] Sending connection context
0927 23:00:49.934210 (+ 180us) client_negotiation.cc:240] Negotiation successful
0927 23:00:49.934656 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"thread_start_us":693,"threads_started":1}
I20260927 23:00:49.937384 18105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550049, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:49.954843 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.938198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:42321 (local address 127.0.0.1:59304)
0927 23:00:49.938419 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.938440 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:49.938555 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:49.939320 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:49.939326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:49.939799 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:49.939951 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.939960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.941239 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.941244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.942621 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.942633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.943719 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.943728 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.943956 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.944384 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:49.944411 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:49.947213 (+ 2802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:49.951640 (+ 4427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.951647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.951650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.952310 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.952832 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.952837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.952840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.953022 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.953736 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:49.953741 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:49.953970 (+ 229us) client_negotiation.cc:769] Sending connection context
0927 23:00:49.954142 (+ 172us) client_negotiation.cc:240] Negotiation successful
0927 23:00:49.954458 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260927 23:00:49.955053 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.938347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.4.74.189:42321)
0927 23:00:49.938968 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.938973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.938992 (+ 19us) server_negotiation.cc:407] Connection header received
0927 23:00:49.939055 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.939061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.939141 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.939249 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:49.940110 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.941088 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.942760 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.943459 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.943685 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.947471 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:49.947477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:49.947481 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:49.947521 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:49.951302 (+ 3781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.952475 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.952483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.952489 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.952677 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.953186 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.953190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.953194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.953564 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:49.953733 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:49.954272 (+ 539us) server_negotiation.cc:299] Negotiation successful
0927 23:00:49.954509 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":368,"threads_started":1}
I20260927 23:00:49.956030 18105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:00:49.956153 18109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.936939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43024 (local address 127.4.74.190:36721)
0927 23:00:49.937528 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.937533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.937547 (+ 14us) server_negotiation.cc:407] Connection header received
0927 23:00:49.937626 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.937631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.937742 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.937851 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:49.939621 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.940642 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.942526 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.943157 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.943308 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.948885 (+ 5577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:49.948893 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:49.948898 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:49.948949 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:49.952421 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.953426 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.953431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.953434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.953592 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.954070 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.954074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.954076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.954492 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:49.954806 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:49.955511 (+ 705us) server_negotiation.cc:299] Negotiation successful
0927 23:00:49.955866 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456}
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550049, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
Sep 27 23:00:49 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
I20260927 23:00:49.972641 18108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.957142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55008 (local address 127.4.74.188:41723)
0927 23:00:49.957370 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.957383 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.957400 (+ 17us) server_negotiation.cc:407] Connection header received
0927 23:00:49.957444 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.957453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.957515 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.957638 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:49.958920 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.959774 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.960977 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.961437 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.961583 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.965562 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:49.965567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:49.965572 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:49.965607 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:49.969401 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.970169 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.970176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.970180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.970302 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.970667 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.970675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.970678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.971228 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:49.971444 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:49.971918 (+ 474us) server_negotiation.cc:299] Negotiation successful
0927 23:00:49.972186 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260927 23:00:49.972781 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.956794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:55008)
0927 23:00:49.957059 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:49.957082 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:49.957197 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:49.957732 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:49.957738 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:49.958513 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:49.958763 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.958775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.959903 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.959910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.960831 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:49.960842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.961603 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.961608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:49.961795 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.962488 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:49.962506 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:49.965303 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:49.969578 (+ 4275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.969585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.969590 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.970022 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.970417 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:49.970424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:49.970428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:49.970564 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:49.971407 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:49.971412 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:49.971651 (+ 239us) client_negotiation.cc:769] Sending connection context
0927 23:00:49.971836 (+ 185us) client_negotiation.cc:240] Negotiation successful
0927 23:00:49.972394 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":141}
I20260927 23:00:49.974695 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.958402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59320 (local address 127.4.74.189:42321)
0927 23:00:49.958608 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.958613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.958828 (+ 215us) server_negotiation.cc:407] Connection header received
0927 23:00:49.959063 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.959069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.959142 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.959245 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:49.960582 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.961302 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.962558 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.962960 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.963096 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:49.968340 (+ 5244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:49.968345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:49.968349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:49.968388 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:49.971323 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.972179 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.972186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.972194 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.972331 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:49.972731 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:49.972740 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:49.972742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:49.973125 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:49.973414 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:49.973998 (+ 584us) server_negotiation.cc:299] Negotiation successful
0927 23:00:49.974298 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 23:00:49.979460 18105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec: Bootstrap starting.
I20260927 23:00:49.981415 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:00:49.983174 18105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:49.984356 18105 log.cc:824] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:49.988895 18105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec: No bootstrap required, opened a new log
I20260927 23:00:50.002672 18105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } }
I20260927 23:00:50.003387 18105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:50.003660 18105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ccaf9ec8f6c46c4a4b11cb57bda41ec, State: Initialized, Role: FOLLOWER
I20260927 23:00:50.004628 18105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } }
I20260927 23:00:50.012159 18120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } } }
I20260927 23:00:50.015306 18120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:00:50.015136 18105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:00:50.015758 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.990226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.4.74.189:42321)
0927 23:00:49.990475 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.990485 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.991673 (+ 1188us) server_negotiation.cc:407] Connection header received
0927 23:00:49.992018 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.992024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.992129 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.992212 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:49.994992 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.996264 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.998732 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:49.999452 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:49.999585 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:50.004479 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:50.004487 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:50.004492 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:50.004548 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:50.010461 (+ 5913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:50.012333 (+ 1872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:50.012341 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:50.012345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:50.012542 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:50.013067 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:50.013074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:50.013077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:50.013516 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:50.013784 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:50.015031 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0927 23:00:50.015306 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260927 23:00:50.019526 18108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.991809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.4.74.188:41723)
0927 23:00:49.992115 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.992124 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.999016 (+ 6892us) server_negotiation.cc:407] Connection header received
0927 23:00:49.999382 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.999387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.999483 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.999600 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:50.002049 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.003136 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.005499 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.006279 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.006365 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:50.010809 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:50.010816 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:50.010821 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:50.010871 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:50.015027 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:50.016145 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:50.016152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:50.016156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:50.016330 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:50.016898 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:50.016903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:50.016906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:50.017361 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:50.017719 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:50.018566 (+ 847us) server_negotiation.cc:299] Negotiation successful
0927 23:00:50.018844 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":98}
I20260927 23:00:50.020000 18109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:49.990336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43032 (local address 127.4.74.190:36721)
0927 23:00:49.990571 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:49.990578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:49.998278 (+ 7700us) server_negotiation.cc:407] Connection header received
0927 23:00:49.998947 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:49.998957 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:49.999057 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:49.999133 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:50.001670 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.002838 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.005554 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.006261 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.006365 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:50.010812 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:50.010818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:50.010822 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:50.010890 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:50.016461 (+ 5571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:50.017593 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:50.017599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:50.017603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:50.017762 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:50.018157 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:50.018162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:50.018166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:50.018536 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:50.018799 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:50.019347 (+ 548us) server_negotiation.cc:299] Negotiation successful
0927 23:00:50.019620 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260927 23:00:50.034861 18131 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:00:50.035040 18131 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:00:50.027690 18038 master_runner.cc:186] Error getting master registration for 127.4.74.190:36721: OK, instance_id { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" instance_seqno: 1790550048672483 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 23:00:50.037642 18038 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 23:00:50.049099 18030 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } attempt: 5
Sep 27 23:00:50 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550049, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
I20260927 23:00:50.066457 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:50.049904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:55028)
0927 23:00:50.050190 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:50.050240 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:50.050407 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:50.050862 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:50.050865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:50.051417 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:50.051680 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:50.051691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.053323 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.053329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:50.054544 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:50.054557 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.055386 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.055391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:50.055648 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:50.056378 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:50.056396 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:50.058960 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:50.063394 (+ 4434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:50.063402 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:50.063407 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:50.064049 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:50.064550 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:50.064556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:50.064560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:50.064739 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:50.065334 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:50.065340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:50.065531 (+ 191us) client_negotiation.cc:769] Sending connection context
0927 23:00:50.065700 (+ 169us) client_negotiation.cc:240] Negotiation successful
0927 23:00:50.066021 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260927 23:00:50.066473 18108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:50.050239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55028 (local address 127.4.74.188:41723)
0927 23:00:50.050473 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:50.050480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:50.050505 (+ 25us) server_negotiation.cc:407] Connection header received
0927 23:00:50.050576 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:50.050582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:50.050702 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:50.050834 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:50.051939 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.053144 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.054700 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.055228 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.055377 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:50.059202 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:50.059210 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:50.059218 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:50.059282 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:50.063150 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:50.064201 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:50.064206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:50.064209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:50.064421 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:50.064867 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:50.064870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:50.064872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:50.065207 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:50.065337 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:50.065891 (+ 554us) server_negotiation.cc:299] Negotiation successful
0927 23:00:50.066149 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 23:00:50.067267 18120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:00:50.067706 18120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } }
I20260927 23:00:50.069334 18120 leader_election.cc:288] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f4dca9310ce404ead56836f7f6fd398 (127.4.74.190:36721), 4a9dd2404eac410b93b33a610ce7ebcd (127.4.74.189:42321)
I20260927 23:00:50.073063 18030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd: Bootstrap starting.
I20260927 23:00:50.077584 18030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:50.079077 18030 log.cc:824] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:50.083156 18030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd: No bootstrap required, opened a new log
I20260927 23:00:50.094710 18030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } }
I20260927 23:00:50.095186 18030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:50.095319 18030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a9dd2404eac410b93b33a610ce7ebcd, State: Initialized, Role: FOLLOWER
I20260927 23:00:50.096078 18030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } }
I20260927 23:00:50.097558 18134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } } }
I20260927 23:00:50.098444 18134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:00:50.098783 18030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:00:50.099175 18005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" is_pre_election: true
I20260927 23:00:50.100203 18005 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ccaf9ec8f6c46c4a4b11cb57bda41ec in term 0.
I20260927 23:00:50.101171 18054 leader_election.cc:302] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ccaf9ec8f6c46c4a4b11cb57bda41ec, 4a9dd2404eac410b93b33a610ce7ebcd; no voters:
I20260927 23:00:50.101866 18120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:00:50.102038 18120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:00:50.102159 18120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:50.111428 18120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } }
I20260927 23:00:50.113404 18004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a9dd2404eac410b93b33a610ce7ebcd"
I20260927 23:00:50.113821 18004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:50.116143 17956 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } attempt: 5
I20260927 23:00:50.116312 18120 leader_election.cc:288] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [CANDIDATE]: Term 1 election: Requested vote from peers 3f4dca9310ce404ead56836f7f6fd398 (127.4.74.190:36721), 4a9dd2404eac410b93b33a610ce7ebcd (127.4.74.189:42321)
I20260927 23:00:50.120940 18004 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ccaf9ec8f6c46c4a4b11cb57bda41ec in term 1.
I20260927 23:00:50.121896 18054 leader_election.cc:302] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ccaf9ec8f6c46c4a4b11cb57bda41ec, 4a9dd2404eac410b93b33a610ce7ebcd; no voters:
I20260927 23:00:50.122619 18120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 1 FOLLOWER]: Leader election won for term 1
W20260927 23:00:50.123468 18059 tablet.cc:2369] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:00:50.128922 18120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [term 1 LEADER]: Becoming Leader. State: Replica: 3ccaf9ec8f6c46c4a4b11cb57bda41ec, State: Running, Role: LEADER
I20260927 23:00:50.129834 18120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } }
Sep 27 23:00:50 dist-test-slave-5pd3 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550048, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
I20260927 23:00:50.146835 18139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ccaf9ec8f6c46c4a4b11cb57bda41ec. Latest consensus state: current_term: 1 leader_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } } }
I20260927 23:00:50.148083 18139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:50.153604 18142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:00:50.154444 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:50.117128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:41723 (local address 127.0.0.1:55030)
0927 23:00:50.117667 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:50.117697 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:50.117872 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:50.125741 (+ 7869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:50.125748 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:50.126667 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:50.126970 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:50.126987 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.133204 (+ 6217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.133218 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:50.134958 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:50.134989 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.136522 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.136531 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:50.136726 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:50.137759 (+ 1033us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:50.137788 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:50.141147 (+ 3359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:50.150736 (+ 9589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:50.150744 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:50.150749 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:50.151468 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:50.152109 (+ 641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:50.152117 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:50.152123 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:50.152349 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:50.153174 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:50.153180 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:50.153407 (+ 227us) client_negotiation.cc:769] Sending connection context
0927 23:00:50.153573 (+ 166us) client_negotiation.cc:240] Negotiation successful
0927 23:00:50.153948 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371}
I20260927 23:00:50.156337 18108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:50.118780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55030 (local address 127.4.74.188:41723)
0927 23:00:50.125251 (+ 6471us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:50.125258 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:50.125286 (+ 28us) server_negotiation.cc:407] Connection header received
0927 23:00:50.125360 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:50.125369 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:50.125483 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:50.127190 (+ 1707us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:50.127377 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.128632 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.135319 (+ 6687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:50.136310 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:50.143720 (+ 7410us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:50.143989 (+ 269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:50.143996 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:50.144006 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:50.144039 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:50.150512 (+ 6473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:50.151703 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:50.151711 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:50.151717 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:50.151965 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:50.152509 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:50.152514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:50.152517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:50.153019 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:50.155361 (+ 2342us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:50.155515 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 23:00:50.155847 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6290}
I20260927 23:00:50.160172 18142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:00:50.168433 17956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398: Bootstrap starting.
I20260927 23:00:50.174535 17956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:50.176714 17956 log.cc:824] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:50.186264 18150 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 23:00:50.191608 18004 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [term 1 FOLLOWER]: Refusing update from remote peer 3ccaf9ec8f6c46c4a4b11cb57bda41ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:00:50.192878 18139 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:00:50.196161 17956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398: No bootstrap required, opened a new log
I20260927 23:00:50.190939 17963 master_runner.cc:186] Error getting master registration for 127.4.74.190:36721: OK, instance_id { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" instance_seqno: 1790550048672483 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 23:00:50.206897 17963 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 23:00:50.208324 17956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } }
I20260927 23:00:50.208999 17956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:50.209218 17956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f4dca9310ce404ead56836f7f6fd398, State: Initialized, Role: FOLLOWER
I20260927 23:00:50.210379 17956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } }
I20260927 23:00:50.220100 18134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ccaf9ec8f6c46c4a4b11cb57bda41ec. Latest consensus state: current_term: 1 leader_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } } }
I20260927 23:00:50.220456 18134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:00:50.221560 17956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:00:50.222332 17929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:50.221632 17931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f4dca9310ce404ead56836f7f6fd398" is_pre_election: true
I20260927 23:00:50.225271 17930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f4dca9310ce404ead56836f7f6fd398"
I20260927 23:00:50.227368 18120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } } }
I20260927 23:00:50.227843 18120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:50.229393 17929 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [term 1 FOLLOWER]: Refusing update from remote peer 3ccaf9ec8f6c46c4a4b11cb57bda41ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:00:50.216559 18152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } } }
I20260927 23:00:50.230521 18152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:00:50.230443 18139 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:00:50.232474 18142 catalog_manager.cc:1347] Generated new cluster ID: ee4513abb8544f8fb8a92583dcc39d6d
I20260927 23:00:50.232589 18142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:00:50.240734 18134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } } }
I20260927 23:00:50.241272 18134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:00:50.271534 18142 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:00:50.273873 18142 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:00:50.286154 18152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ccaf9ec8f6c46c4a4b11cb57bda41ec. Latest consensus state: current_term: 1 leader_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } } }
I20260927 23:00:50.286636 18152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:00:50.287078 18161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } } }
I20260927 23:00:50.287516 18161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:00:50.288509 18120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f4dca9310ce404ead56836f7f6fd398" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36721 } } peers { permanent_uuid: "4a9dd2404eac410b93b33a610ce7ebcd" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 42321 } } peers { permanent_uuid: "3ccaf9ec8f6c46c4a4b11cb57bda41ec" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 41723 } } }
I20260927 23:00:50.288877 18120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:50.296677 18142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ccaf9ec8f6c46c4a4b11cb57bda41ec: Generated new TSK 0
I20260927 23:00:50.297891 18142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:00:50.335408 18171 catalog_manager.cc:1259] Loaded cluster ID: ee4513abb8544f8fb8a92583dcc39d6d
I20260927 23:00:50.347572 18171 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398: loading cluster ID for follower catalog manager: success
I20260927 23:00:50.361620 18171 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398: acquiring CA information for follower catalog manager: success
I20260927 23:00:50.368158 18171 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f4dca9310ce404ead56836f7f6fd398: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:00:51.191362 18150 catalog_manager.cc:1259] Loaded cluster ID: ee4513abb8544f8fb8a92583dcc39d6d
I20260927 23:00:51.191529 18150 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd: loading cluster ID for follower catalog manager: success
I20260927 23:00:51.194521 18150 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd: acquiring CA information for follower catalog manager: success
I20260927 23:00:51.196722 18150 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4a9dd2404eac410b93b33a610ce7ebcd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:00:51.467569 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 17889
I20260927 23:00:51.469452 17951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:51.601528 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 17889
I20260927 23:00:51.627225 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 17963
I20260927 23:00:51.628633 18025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:51.745797 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 17963
I20260927 23:00:51.767680 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 18038
I20260927 23:00:51.768968 18100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:51.883190 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 18038
2026-09-27T23:00:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5126 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:00:51 dist-test-slave-5pd3 krb5kdc[18182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:00:51 dist-test-slave-5pd3 krb5kdc[18182](info): set up 2 sockets
Sep 27 23:00:51 dist-test-slave-5pd3 krb5kdc[18182](info): commencing operation
krb5kdc: starting...
W20260927 23:00:53.020277 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:00:53 dist-test-slave-5pd3 krb5kdc[18182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:00:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:00:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:00:53.299777 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36899
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:34049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:00:53.706984 18195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:53.707392 18195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:53.707620 18195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:53.715796 18195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:00:53.715953 18195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:53.716079 18195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:53.716176 18195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:00:53.716287 18195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:00:53.728488 18195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:34049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:53.730275 18195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:53.731997 18195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:53.746610 18201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:53.746877 18200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:53.747530 18203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:53.748101 18195 server_base.cc:1034] running on GCE node
I20260927 23:00:53.749078 18195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:53.750837 18195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:53.752111 18195 hybrid_clock.cc:648] HybridClock initialized: now 1790550053752046 us; error 61 us; skew 500 ppm
I20260927 23:00:53.752530 18195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:53 dist-test-slave-5pd3 krb5kdc[18182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550053, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:53.762846 18195 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:00:53.765141 18195 webserver.cc:460] Webserver started at http://127.4.74.190:34689/ using document root <none> and password file <none>
I20260927 23:00:53.766028 18195 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:53.766187 18195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:53.766544 18195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:00:53.770053 18195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "623b0f7759de41c8a635b290e48c7253"
format_stamp: "Formatted at 2026-09-27 23:00:53 on dist-test-slave-5pd3"
server_key: "2ce229cd23c47d3d2c65c73eea49a3b4"
server_key_iv: "1e7aa9fe94df1ff6c874f44990de7488"
server_key_version: "encryptionkey@0"
I20260927 23:00:53.770973 18195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "623b0f7759de41c8a635b290e48c7253"
format_stamp: "Formatted at 2026-09-27 23:00:53 on dist-test-slave-5pd3"
server_key: "2ce229cd23c47d3d2c65c73eea49a3b4"
server_key_iv: "1e7aa9fe94df1ff6c874f44990de7488"
server_key_version: "encryptionkey@0"
I20260927 23:00:53.777518 18195 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260927 23:00:53.781829 18210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:53.783766 18195 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:00:53.783989 18195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "623b0f7759de41c8a635b290e48c7253"
format_stamp: "Formatted at 2026-09-27 23:00:53 on dist-test-slave-5pd3"
server_key: "2ce229cd23c47d3d2c65c73eea49a3b4"
server_key_iv: "1e7aa9fe94df1ff6c874f44990de7488"
server_key_version: "encryptionkey@0"
I20260927 23:00:53.784222 18195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:53.820016 18195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:53.824712 18195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:53.825029 18195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:53.846690 18195 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:36899
I20260927 23:00:53.846805 18261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:36899 every 8 connection(s)
I20260927 23:00:53.848330 18195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:00:53.849107 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 18195
I20260927 23:00:53.849408 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:00:53.850382 4394 external_mini_cluster.cc:1428] Setting key 06c803e709ee5717064fed14c063899e
I20260927 23:00:53.853295 18262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:53.867419 18262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253: Bootstrap starting.
Sep 27 23:00:53 dist-test-slave-5pd3 krb5kdc[18182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:53.871466 18262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:53.872745 18262 log.cc:824] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:53.876597 18262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253: No bootstrap required, opened a new log
I20260927 23:00:53.880177 18265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:53.852947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38970 (local address 127.4.74.190:36899)
0927 23:00:53.853995 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:53.854022 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:53.854080 (+ 58us) server_negotiation.cc:407] Connection header received
0927 23:00:53.856570 (+ 2490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:53.856600 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:53.857173 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:53.857672 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:53.859387 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:53.861503 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:53.863144 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:53.864295 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:53.864526 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:53.870257 (+ 5731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:53.870268 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:53.870290 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:53.870357 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:53.874310 (+ 3953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:53.875604 (+ 1294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:53.875623 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:53.875634 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:53.875866 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:53.876383 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:53.876387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:53.876389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:53.877164 (+ 775us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:53.877474 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:53.878033 (+ 559us) server_negotiation.cc:299] Negotiation successful
0927 23:00:53.878642 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":458,"threads_started":1}
I20260927 23:00:53.886093 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:00:53.886960 18262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0f7759de41c8a635b290e48c7253" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36899 } }
I20260927 23:00:53.887408 18262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:53.887524 18262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 623b0f7759de41c8a635b290e48c7253, State: Initialized, Role: FOLLOWER
I20260927 23:00:53.888270 18262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0f7759de41c8a635b290e48c7253" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36899 } }
I20260927 23:00:53.888527 18262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:00:53.888661 18262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:00:53.888854 18262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:53.892910 18262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0f7759de41c8a635b290e48c7253" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36899 } }
I20260927 23:00:53.893567 18262 leader_election.cc:302] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 623b0f7759de41c8a635b290e48c7253; no voters:
I20260927 23:00:53.894359 18262 leader_election.cc:288] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:00:53.894678 18267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:00:53.895700 18267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [term 1 LEADER]: Becoming Leader. State: Replica: 623b0f7759de41c8a635b290e48c7253, State: Running, Role: LEADER
I20260927 23:00:53.896443 18267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0f7759de41c8a635b290e48c7253" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36899 } }
I20260927 23:00:53.897136 18262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:00:53.900818 18268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "623b0f7759de41c8a635b290e48c7253" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0f7759de41c8a635b290e48c7253" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36899 } } }
I20260927 23:00:53.901443 18268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:53.902410 18269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 623b0f7759de41c8a635b290e48c7253. Latest consensus state: current_term: 1 leader_uuid: "623b0f7759de41c8a635b290e48c7253" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0f7759de41c8a635b290e48c7253" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36899 } } }
I20260927 23:00:53.902921 18274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:00:53.902921 18269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253 [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:53.910079 18274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:00:53.922520 18274 catalog_manager.cc:1347] Generated new cluster ID: d1bfd58df92243f297c83da7ce96e873
I20260927 23:00:53.922722 18274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:00:53.943815 18274 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:00:53.945304 18274 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:00:53.956975 18274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 623b0f7759de41c8a635b290e48c7253: Generated new TSK 0
I20260927 23:00:53.957937 18274 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:00:54.001552 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 18195
I20260927 23:00:54.002974 18257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:54.119422 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 18195
2026-09-27T23:00:54Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2237 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:00:54 dist-test-slave-5pd3 krb5kdc[18290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:00:54 dist-test-slave-5pd3 krb5kdc[18290](info): set up 2 sockets
Sep 27 23:00:54 dist-test-slave-5pd3 krb5kdc[18290](info): commencing operation
krb5kdc: starting...
W20260927 23:00:55.288578 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:00:55 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:00:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:00:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:00:55.563964 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:43367
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:43367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:00:55.987386 18303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:55.987980 18303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:55.988163 18303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:55.997042 18303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:00:55.997267 18303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:55.997397 18303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:55.997512 18303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:00:55.997596 18303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:00:56.011018 18303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:43367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:43367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:56.013091 18303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:56.015048 18303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:56.032246 18309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:56.032464 18308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:56.032444 18311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:56.033874 18303 server_base.cc:1034] running on GCE node
I20260927 23:00:56.034924 18303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:56.036921 18303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:56.038173 18303 hybrid_clock.cc:648] HybridClock initialized: now 1790550056038116 us; error 44 us; skew 500 ppm
I20260927 23:00:56.038635 18303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:56 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550056, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:56.042876 18303 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:00:56.044986 18303 webserver.cc:460] Webserver started at http://127.4.74.190:45127/ using document root <none> and password file <none>
I20260927 23:00:56.045898 18303 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:56.046046 18303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:56.046446 18303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:00:56.050411 18303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "08b5464f552447d7832812283a3399fd"
format_stamp: "Formatted at 2026-09-27 23:00:56 on dist-test-slave-5pd3"
server_key: "1cce4809e607ffe9d11b0effe846ae91"
server_key_iv: "dda73a9758a91e300a72200af05129db"
server_key_version: "encryptionkey@0"
I20260927 23:00:56.051445 18303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "08b5464f552447d7832812283a3399fd"
format_stamp: "Formatted at 2026-09-27 23:00:56 on dist-test-slave-5pd3"
server_key: "1cce4809e607ffe9d11b0effe846ae91"
server_key_iv: "dda73a9758a91e300a72200af05129db"
server_key_version: "encryptionkey@0"
I20260927 23:00:56.058297 18303 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:00:56.063092 18318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:56.065188 18303 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:00:56.065452 18303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "08b5464f552447d7832812283a3399fd"
format_stamp: "Formatted at 2026-09-27 23:00:56 on dist-test-slave-5pd3"
server_key: "1cce4809e607ffe9d11b0effe846ae91"
server_key_iv: "dda73a9758a91e300a72200af05129db"
server_key_version: "encryptionkey@0"
I20260927 23:00:56.065727 18303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:56.081168 18303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:56.086596 18303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:56.086970 18303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:56.112479 18303 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:43367
I20260927 23:00:56.112505 18369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:43367 every 8 connection(s)
I20260927 23:00:56.114319 18303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:00:56.118762 18370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:56.123602 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 18303
I20260927 23:00:56.123905 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:00:56.124773 4394 external_mini_cluster.cc:1428] Setting key 36e46223cc2dd5c3fb3124d5c26c84bb
I20260927 23:00:56.131798 18370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Bootstrap starting.
I20260927 23:00:56.135610 18370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:56.136865 18370 log.cc:824] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:56.141106 18370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: No bootstrap required, opened a new log
Sep 27 23:00:56 dist-test-slave-5pd3 krb5kdc[18290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:56.152809 18370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:00:56.153365 18370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:56.153518 18370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08b5464f552447d7832812283a3399fd, State: Initialized, Role: FOLLOWER
I20260927 23:00:56.154305 18370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:00:56.154639 18370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:00:56.154815 18370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:00:56.155023 18370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:56.156730 18373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:56.127721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.4.74.190:43367)
0927 23:00:56.128931 (+ 1210us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:56.128949 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:56.129015 (+ 66us) server_negotiation.cc:407] Connection header received
0927 23:00:56.131516 (+ 2501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:56.131577 (+ 61us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:56.132141 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:56.132617 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:56.134284 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:56.136367 (+ 2083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:56.138690 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:56.139954 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:56.140203 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:56.145515 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:56.145527 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:56.145550 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:56.145607 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:56.150561 (+ 4954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:56.151779 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:56.151791 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:56.151802 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:56.152065 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:56.152620 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:56.152625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:56.152629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:56.153544 (+ 915us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:56.153885 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:56.154546 (+ 661us) server_negotiation.cc:299] Negotiation successful
0927 23:00:56.155129 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":897,"thread_start_us":608,"threads_started":1}
I20260927 23:00:56.158366 18370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:00:56.158980 18370 leader_election.cc:302] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08b5464f552447d7832812283a3399fd; no voters:
I20260927 23:00:56.159746 18370 leader_election.cc:288] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:00:56.159953 18375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:00:56.162106 18375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 1 LEADER]: Becoming Leader. State: Replica: 08b5464f552447d7832812283a3399fd, State: Running, Role: LEADER
I20260927 23:00:56.162238 18370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:00:56.162904 18375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:00:56.167603 18377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08b5464f552447d7832812283a3399fd. Latest consensus state: current_term: 1 leader_uuid: "08b5464f552447d7832812283a3399fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } } }
I20260927 23:00:56.167421 18376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08b5464f552447d7832812283a3399fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } } }
I20260927 23:00:56.169078 18376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:56.169544 18377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:56.176559 18384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:00:56.183897 18384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:00:56.195245 18384 catalog_manager.cc:1347] Generated new cluster ID: ca08be00a6014afea7d21840330483fa
I20260927 23:00:56.195398 18384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:00:56.216862 18384 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:00:56.218706 18384 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:00:56.231429 18384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Generated new TSK 0
I20260927 23:00:56.236811 18384 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 23:00:56.321077 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43367
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:00:56.743881 18396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:56.744288 18396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:56.744472 18396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:56.753561 18396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:56.753727 18396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:56.753968 18396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:00:56.765870 18396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:56.767859 18396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:56.769613 18396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:56.787598 18401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:56.787940 18402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:56.788136 18404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:56.789495 18396 server_base.cc:1034] running on GCE node
I20260927 23:00:56.790510 18396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:56.791991 18396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:56.793280 18396 hybrid_clock.cc:648] HybridClock initialized: now 1790550056793225 us; error 48 us; skew 500 ppm
I20260927 23:00:56.793747 18396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:56 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550056, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:56.798058 18396 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:00:56.800288 18396 webserver.cc:460] Webserver started at http://127.4.74.129:33317/ using document root <none> and password file <none>
I20260927 23:00:56.801268 18396 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:56.801447 18396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:56.801837 18396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:00:56.806039 18396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "0eac920703e24a4c8b719e86415da811"
format_stamp: "Formatted at 2026-09-27 23:00:56 on dist-test-slave-5pd3"
server_key: "88a67e1c89beec43edce4c38438b5be5"
server_key_iv: "64e77e4f85576aa97e670a5695425631"
server_key_version: "encryptionkey@0"
I20260927 23:00:56.807296 18396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "0eac920703e24a4c8b719e86415da811"
format_stamp: "Formatted at 2026-09-27 23:00:56 on dist-test-slave-5pd3"
server_key: "88a67e1c89beec43edce4c38438b5be5"
server_key_iv: "64e77e4f85576aa97e670a5695425631"
server_key_version: "encryptionkey@0"
I20260927 23:00:56.814954 18396 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260927 23:00:56.820122 18411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:56.822276 18396 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:00:56.822561 18396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "0eac920703e24a4c8b719e86415da811"
format_stamp: "Formatted at 2026-09-27 23:00:56 on dist-test-slave-5pd3"
server_key: "88a67e1c89beec43edce4c38438b5be5"
server_key_iv: "64e77e4f85576aa97e670a5695425631"
server_key_version: "encryptionkey@0"
I20260927 23:00:56.822866 18396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:56.838614 18396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:56.844492 18396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:56.844866 18396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:56.846150 18396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:00:56.848071 18396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:00:56.848210 18396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:56.848388 18396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:00:56.848469 18396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:56.894451 18396 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:44205
I20260927 23:00:56.894516 18523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:44205 every 8 connection(s)
I20260927 23:00:56.896492 18396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:00:56.903460 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 18396
I20260927 23:00:56.903782 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 23:00:56.904500 4394 external_mini_cluster.cc:1428] Setting key a28c5436a394c669c7e4661269a171cf
Sep 27 23:00:56 dist-test-slave-5pd3 krb5kdc[18290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550056, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:56.924240 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:56.900029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:43229 (local address 127.4.74.190:43367)
0927 23:00:56.900617 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:56.900624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:56.901809 (+ 1185us) server_negotiation.cc:407] Connection header received
0927 23:00:56.904309 (+ 2500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:56.904316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:56.904436 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:56.904579 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:56.906968 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:56.907876 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:56.909983 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:56.910464 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:56.910630 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:56.915340 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:56.915346 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:56.915350 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:56.915395 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:56.918752 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:56.920028 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:56.920039 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:56.920043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:56.920222 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:56.921231 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:56.921235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:56.921238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:56.921656 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:56.922301 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:56.923348 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0927 23:00:56.923779 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":298,"threads_started":1}
I20260927 23:00:56.925586 18527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:56.900592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:43229)
0927 23:00:56.901494 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:56.901553 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:56.903894 (+ 2341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:56.904913 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:56.904925 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:56.905625 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:56.906743 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:56.906774 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:56.908067 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:56.908072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:56.909745 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:56.909765 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:56.910763 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:56.910772 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:56.911070 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:56.912228 (+ 1158us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:56.912258 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:56.914997 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:56.919038 (+ 4041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:56.919060 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:56.919082 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:56.919844 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:56.920483 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:56.920491 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:56.920536 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:56.920935 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:56.922108 (+ 1173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:56.922122 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:56.922801 (+ 679us) client_negotiation.cc:769] Sending connection context
0927 23:00:56.923194 (+ 393us) client_negotiation.cc:240] Negotiation successful
0927 23:00:56.924034 (+ 840us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":338,"threads_started":1}
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 23:00:56.928284 18524 heartbeater.cc:347] Connected to a master server at 127.4.74.190:43367
I20260927 23:00:56.928874 18524 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:56.930133 18524 heartbeater.cc:511] Master 127.4.74.190:43367 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
I20260927 23:00:56.933947 18335 ts_manager.cc:194] Registered new tserver with Master: 0eac920703e24a4c8b719e86415da811 (127.4.74.129:44205)
I20260927 23:00:56.936625 18335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:43229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 23:00:56.967248 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43367
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:00:57.338146 18530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:57.338456 18530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:57.338554 18530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:57.345918 18530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:57.346063 18530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:57.346184 18530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:00:57.358058 18530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:57.359409 18530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:57.360821 18530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:57.375691 18535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:57.376204 18538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:57.376266 18536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:57.381349 18530 server_base.cc:1034] running on GCE node
I20260927 23:00:57.382159 18530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:57.383276 18530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:57.384620 18530 hybrid_clock.cc:648] HybridClock initialized: now 1790550057384562 us; error 58 us; skew 500 ppm
I20260927 23:00:57.385025 18530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:57 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550057, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:57.388612 18530 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:00:57.390484 18530 webserver.cc:460] Webserver started at http://127.4.74.130:33519/ using document root <none> and password file <none>
I20260927 23:00:57.391312 18530 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:57.391487 18530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:57.391882 18530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:00:57.395095 18530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "db08bb386fa447cbb7f5a36eaac6b648"
format_stamp: "Formatted at 2026-09-27 23:00:57 on dist-test-slave-5pd3"
server_key: "4782a582dff250eb8edd8b1febcd57ae"
server_key_iv: "533b9ef41b636a2d99c7aa642ed86f2c"
server_key_version: "encryptionkey@0"
I20260927 23:00:57.396127 18530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "db08bb386fa447cbb7f5a36eaac6b648"
format_stamp: "Formatted at 2026-09-27 23:00:57 on dist-test-slave-5pd3"
server_key: "4782a582dff250eb8edd8b1febcd57ae"
server_key_iv: "533b9ef41b636a2d99c7aa642ed86f2c"
server_key_version: "encryptionkey@0"
I20260927 23:00:57.402469 18530 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:00:57.406810 18545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:57.408694 18530 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:00:57.408959 18530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "db08bb386fa447cbb7f5a36eaac6b648"
format_stamp: "Formatted at 2026-09-27 23:00:57 on dist-test-slave-5pd3"
server_key: "4782a582dff250eb8edd8b1febcd57ae"
server_key_iv: "533b9ef41b636a2d99c7aa642ed86f2c"
server_key_version: "encryptionkey@0"
I20260927 23:00:57.409227 18530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:57.465195 18530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:57.470157 18530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:57.470518 18530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:57.471871 18530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:00:57.473585 18530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:00:57.473690 18530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:57.473858 18530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:00:57.473933 18530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:57.513144 18530 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:38959
I20260927 23:00:57.513254 18657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:38959 every 8 connection(s)
I20260927 23:00:57.514649 18530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:00:57.516901 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 18530
I20260927 23:00:57.517117 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 23:00:57.517725 4394 external_mini_cluster.cc:1428] Setting key 6da88fa8f5d87ac1a4f7a135c1e77d84
Sep 27 23:00:57 dist-test-slave-5pd3 krb5kdc[18290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550057, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:57.544548 18660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:57.518275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:35385 (local address 127.4.74.190:43367)
0927 23:00:57.519007 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:57.519016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:57.522827 (+ 3811us) server_negotiation.cc:407] Connection header received
0927 23:00:57.522920 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:57.522929 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:57.523057 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:57.523188 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:57.525475 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:57.526739 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:57.529283 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:57.529736 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:57.529850 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:57.535591 (+ 5741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:57.535603 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:57.535612 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:57.535675 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:57.539840 (+ 4165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:57.540960 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:57.540971 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:57.541084 (+ 113us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:57.541324 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:57.541939 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:57.541948 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:57.541954 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:57.542479 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:57.542676 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:57.543820 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0927 23:00:57.544140 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":388,"threads_started":1}
I20260927 23:00:57.545562 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:57.518939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:35385)
0927 23:00:57.519817 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:57.519865 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:57.522419 (+ 2554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:57.523647 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:57.523661 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:57.524411 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:57.525197 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:57.525220 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:57.526959 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:57.526965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:57.529083 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:57.529106 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:57.530054 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:57.530065 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:57.530654 (+ 589us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:57.531658 (+ 1004us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:57.531705 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:57.535160 (+ 3455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:57.540133 (+ 4973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:57.540153 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:57.540168 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:57.540748 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:57.541533 (+ 785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:57.541537 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:57.541540 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:57.541768 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:57.542706 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:57.542713 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:57.543128 (+ 415us) client_negotiation.cc:769] Sending connection context
0927 23:00:57.543505 (+ 377us) client_negotiation.cc:240] Negotiation successful
0927 23:00:57.544177 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":354,"threads_started":1}
I20260927 23:00:57.547407 18658 heartbeater.cc:347] Connected to a master server at 127.4.74.190:43367
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 23:00:57.547945 18658 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:57.548981 18658 heartbeater.cc:511] Master 127.4.74.190:43367 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
I20260927 23:00:57.551137 18335 ts_manager.cc:194] Registered new tserver with Master: db08bb386fa447cbb7f5a36eaac6b648 (127.4.74.130:38959)
I20260927 23:00:57.552243 18335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:35385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 23:00:57.583437 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43367
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:00:57.941956 18524 heartbeater.cc:503] Master 127.4.74.190:43367 was elected leader, sending a full tablet report...
W20260927 23:00:57.998910 18664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:57.999464 18664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:57.999738 18664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:58.008541 18664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:58.008728 18664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:58.009249 18664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:00:58.023401 18664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:58.025326 18664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:58.027206 18664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:58.044646 18672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:58.045156 18669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:58.044872 18670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:58.046833 18664 server_base.cc:1034] running on GCE node
I20260927 23:00:58.047763 18664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:58.049093 18664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:58.050468 18664 hybrid_clock.cc:648] HybridClock initialized: now 1790550058050404 us; error 48 us; skew 500 ppm
I20260927 23:00:58.050949 18664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:58 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550058, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:58.055116 18664 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:00:58.057451 18664 webserver.cc:460] Webserver started at http://127.4.74.131:43657/ using document root <none> and password file <none>
I20260927 23:00:58.058457 18664 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:58.058635 18664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:58.059046 18664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:00:58.062897 18664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "ab3f836c98184076be1ba35724985bbf"
format_stamp: "Formatted at 2026-09-27 23:00:58 on dist-test-slave-5pd3"
server_key: "49b12116ee47bc679648ea1fc3e543a9"
server_key_iv: "17c958b34c755be7bc3dfc5d96578f1d"
server_key_version: "encryptionkey@0"
I20260927 23:00:58.063908 18664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "ab3f836c98184076be1ba35724985bbf"
format_stamp: "Formatted at 2026-09-27 23:00:58 on dist-test-slave-5pd3"
server_key: "49b12116ee47bc679648ea1fc3e543a9"
server_key_iv: "17c958b34c755be7bc3dfc5d96578f1d"
server_key_version: "encryptionkey@0"
I20260927 23:00:58.070724 18664 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:00:58.075408 18679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:58.077432 18664 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260927 23:00:58.077704 18664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "ab3f836c98184076be1ba35724985bbf"
format_stamp: "Formatted at 2026-09-27 23:00:58 on dist-test-slave-5pd3"
server_key: "49b12116ee47bc679648ea1fc3e543a9"
server_key_iv: "17c958b34c755be7bc3dfc5d96578f1d"
server_key_version: "encryptionkey@0"
I20260927 23:00:58.077977 18664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:58.103453 18664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:58.110477 18664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:58.110855 18664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:58.112422 18664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:00:58.114351 18664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:00:58.114473 18664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:58.114645 18664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:00:58.114717 18664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:58.158377 18664 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:43659
I20260927 23:00:58.158432 18791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:43659 every 8 connection(s)
I20260927 23:00:58.160108 18664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:00:58.163914 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 18664
I20260927 23:00:58.164177 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 23:00:58.164716 4394 external_mini_cluster.cc:1428] Setting key 639b0b3cc46d964dbc62c035e9cf6983
Sep 27 23:00:58 dist-test-slave-5pd3 krb5kdc[18290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550058, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:00:58.187582 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.163773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:43697 (local address 127.4.74.190:43367)
0927 23:00:58.164433 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.164440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.165752 (+ 1312us) server_negotiation.cc:407] Connection header received
0927 23:00:58.168769 (+ 3017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.168776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.168888 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.168994 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:58.171135 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.172152 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.174225 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.174703 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.174874 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.180276 (+ 5402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:58.180282 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:58.180286 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:58.180331 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:58.184054 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:58.184981 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:58.184987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:58.184991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:58.185132 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:58.185594 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:58.185600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:58.185603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:58.185908 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:58.186086 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.186875 (+ 789us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.187129 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":276,"threads_started":1}
I20260927 23:00:58.188683 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.164351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:43697)
0927 23:00:58.165496 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:58.165566 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:58.168268 (+ 2702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:58.169345 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:58.169356 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:58.170061 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:00:58.170895 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.170925 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.172360 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.172368 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.173867 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.173882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.175174 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.175187 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.175660 (+ 473us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.176802 (+ 1142us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:00:58.176838 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:00:58.179939 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:00:58.184257 (+ 4318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:58.184269 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:58.184296 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:58.184836 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:58.185256 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:00:58.185261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:00:58.185264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:00:58.185473 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:00:58.186081 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:00:58.186089 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:00:58.186491 (+ 402us) client_negotiation.cc:769] Sending connection context
0927 23:00:58.186783 (+ 292us) client_negotiation.cc:240] Negotiation successful
0927 23:00:58.187394 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":379,"threads_started":1}
I20260927 23:00:58.191044 18792 heartbeater.cc:347] Connected to a master server at 127.4.74.190:43367
I20260927 23:00:58.191586 18792 heartbeater.cc:464] Registering TS with master...
I20260927 23:00:58.192521 18792 heartbeater.cc:511] Master 127.4.74.190:43367 requested a full tablet report, sending...
I20260927 23:00:58.194782 18335 ts_manager.cc:194] Registered new tserver with Master: ab3f836c98184076be1ba35724985bbf (127.4.74.131:43659)
I20260927 23:00:58.196038 18335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:43697
I20260927 23:00:58.196585 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:00:58.220876 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.202575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.4.74.190:43367)
0927 23:00:58.202942 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.202954 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.203320 (+ 366us) server_negotiation.cc:407] Connection header received
0927 23:00:58.203638 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.203643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.203741 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.203884 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:58.205807 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.206870 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.208688 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.209176 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.209364 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.212911 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:58.212918 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:58.212922 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:58.213003 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:58.216844 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:58.217946 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:58.217953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:58.217956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:58.218132 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:58.218576 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:58.218584 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:58.218591 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:58.219179 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:58.219529 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.220156 (+ 627us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.220479 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260927 23:00:58.225972 18335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:00:58.233863 18335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:00:58.262826 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.250444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.4.74.130:38959)
0927 23:00:58.251291 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.251303 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.253344 (+ 2041us) server_negotiation.cc:407] Connection header received
0927 23:00:58.253882 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.253913 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.254347 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.254594 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:00:58.255871 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.257592 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.260221 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.261391 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.261581 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.261853 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.262207 (+ 354us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.262444 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":871,"threads_started":1}
I20260927 23:00:58.263525 18808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.250339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.0.0.1:41262)
0927 23:00:58.252105 (+ 1766us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:58.252174 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:58.252437 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:58.255512 (+ 3075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:58.255518 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:58.255569 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:00:58.256044 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.256058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.258289 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.258295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.260222 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.260238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.262207 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.262213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.262450 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.262504 (+ 54us) client_negotiation.cc:769] Sending connection context
0927 23:00:58.262600 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 23:00:58.262866 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1397,"mutex_wait_us":23,"thread_start_us":2479,"threads_started":1}
I20260927 23:00:58.264008 18805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.251041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:38959 (local address 127.0.0.1:35860)
0927 23:00:58.253193 (+ 2152us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:58.253217 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:58.253496 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:58.254874 (+ 1378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:58.254884 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:58.254919 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:00:58.255571 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.255586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.257829 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.257834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.260014 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.260027 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.261624 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.261630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.261865 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.261931 (+ 66us) client_negotiation.cc:769] Sending connection context
0927 23:00:58.263172 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0927 23:00:58.263352 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1969,"mutex_wait_us":37,"thread_start_us":607,"threads_started":1}
I20260927 23:00:58.264688 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.250861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.4.74.129:44205)
0927 23:00:58.254806 (+ 3945us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.254815 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.254844 (+ 29us) server_negotiation.cc:407] Connection header received
0927 23:00:58.254942 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.254950 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.255241 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.256257 (+ 1016us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:00:58.256530 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.258131 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.260398 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.261913 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.263606 (+ 1693us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.263935 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.264065 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.264318 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3741,"thread_start_us":3571,"threads_started":1}
I20260927 23:00:58.268280 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.254083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:43659 (local address 127.0.0.1:34846)
0927 23:00:58.255664 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:58.255725 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:58.255931 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:58.257730 (+ 1799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:58.257734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:58.257758 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:00:58.260167 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.260176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.262358 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.262363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.265129 (+ 2766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.265148 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.267181 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.267188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.267359 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.267397 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 23:00:58.267479 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 23:00:58.267676 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1316,"mutex_wait_us":6,"thread_start_us":1074,"threads_started":1}
I20260927 23:00:58.268791 18593 tablet_service.cc:1467] Processing CreateTablet for tablet b83fb8c3a28546a9a940cfaf3d0768ca (DEFAULT_TABLE table=test-table [id=4204e54cd05245bd85866da9c3a7166c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:00:58.269726 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.253720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34846 (local address 127.4.74.131:43659)
0927 23:00:58.256744 (+ 3024us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.256752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.256782 (+ 30us) server_negotiation.cc:407] Connection header received
0927 23:00:58.256918 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.256926 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.257197 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.257419 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:00:58.260394 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.262181 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.265351 (+ 3170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.266911 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.267113 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.268986 (+ 1873us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.269101 (+ 115us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.269307 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2797,"thread_start_us":2597,"threads_started":1}
I20260927 23:00:58.269328 18459 tablet_service.cc:1467] Processing CreateTablet for tablet b83fb8c3a28546a9a940cfaf3d0768ca (DEFAULT_TABLE table=test-table [id=4204e54cd05245bd85866da9c3a7166c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:00:58.270468 18593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b83fb8c3a28546a9a940cfaf3d0768ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:58.271092 18459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b83fb8c3a28546a9a940cfaf3d0768ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:58.273769 18727 tablet_service.cc:1467] Processing CreateTablet for tablet b83fb8c3a28546a9a940cfaf3d0768ca (DEFAULT_TABLE table=test-table [id=4204e54cd05245bd85866da9c3a7166c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:00:58.275511 18727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b83fb8c3a28546a9a940cfaf3d0768ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:00:58.285535 18811 tablet_bootstrap.cc:492] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: Bootstrap starting.
I20260927 23:00:58.289602 18811 tablet_bootstrap.cc:654] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:58.290849 18812 tablet_bootstrap.cc:492] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: Bootstrap starting.
I20260927 23:00:58.292198 18811 log.cc:824] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:58.293937 18813 tablet_bootstrap.cc:492] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: Bootstrap starting.
I20260927 23:00:58.295853 18812 tablet_bootstrap.cc:654] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:58.297868 18811 tablet_bootstrap.cc:492] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: No bootstrap required, opened a new log
I20260927 23:00:58.297803 18812 log.cc:824] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:58.298269 18811 ts_tablet_manager.cc:1397] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260927 23:00:58.298632 18813 tablet_bootstrap.cc:654] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: Neither blocks nor log segments found. Creating new log.
I20260927 23:00:58.300343 18813 log.cc:824] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:58.302114 18812 tablet_bootstrap.cc:492] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: No bootstrap required, opened a new log
I20260927 23:00:58.302489 18812 ts_tablet_manager.cc:1397] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260927 23:00:58.304181 18813 tablet_bootstrap.cc:492] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: No bootstrap required, opened a new log
I20260927 23:00:58.304512 18813 ts_tablet_manager.cc:1397] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260927 23:00:58.307611 18811 raft_consensus.cc:348] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } }
I20260927 23:00:58.308074 18811 raft_consensus.cc:374] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:58.308180 18811 raft_consensus.cc:729] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eac920703e24a4c8b719e86415da811, State: Initialized, Role: FOLLOWER
I20260927 23:00:58.308836 18811 consensus_queue.cc:260] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } }
I20260927 23:00:58.310642 18811 ts_tablet_manager.cc:1428] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260927 23:00:58.314096 18812 raft_consensus.cc:348] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } }
I20260927 23:00:58.314666 18812 raft_consensus.cc:374] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:58.314801 18812 raft_consensus.cc:729] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db08bb386fa447cbb7f5a36eaac6b648, State: Initialized, Role: FOLLOWER
I20260927 23:00:58.315685 18812 consensus_queue.cc:260] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } }
I20260927 23:00:58.316536 18813 raft_consensus.cc:348] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } }
I20260927 23:00:58.317065 18813 raft_consensus.cc:374] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:00:58.317210 18813 raft_consensus.cc:729] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab3f836c98184076be1ba35724985bbf, State: Initialized, Role: FOLLOWER
I20260927 23:00:58.318006 18813 consensus_queue.cc:260] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } }
I20260927 23:00:58.318475 18658 heartbeater.cc:503] Master 127.4.74.190:43367 was elected leader, sending a full tablet report...
I20260927 23:00:58.319396 18812 ts_tablet_manager.cc:1428] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: Time spent starting tablet: real 0.017s user 0.018s sys 0.000s
I20260927 23:00:58.321789 18792 heartbeater.cc:503] Master 127.4.74.190:43367 was elected leader, sending a full tablet report...
I20260927 23:00:58.322659 18813 ts_tablet_manager.cc:1428] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: Time spent starting tablet: real 0.018s user 0.005s sys 0.011s
W20260927 23:00:58.400106 18525 tablet.cc:2369] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 23:00:58.412248 18793 tablet.cc:2369] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:00:58.482501 18818 raft_consensus.cc:482] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:00:58.482993 18818 raft_consensus.cc:504] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } }
I20260927 23:00:58.485138 18818 leader_election.cc:288] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0eac920703e24a4c8b719e86415da811 (127.4.74.129:44205), ab3f836c98184076be1ba35724985bbf (127.4.74.131:43659)
I20260927 23:00:58.493878 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.485806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:38077 (local address 127.4.74.129:44205)
0927 23:00:58.486047 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.486051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.486529 (+ 478us) server_negotiation.cc:407] Connection header received
0927 23:00:58.486922 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.486928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.487018 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.487093 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:00:58.487939 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.489106 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.491346 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.492508 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.492694 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.492791 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.493356 (+ 565us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.493565 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260927 23:00:58.494280 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.486779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:57033 (local address 127.4.74.131:43659)
0927 23:00:58.487018 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.487027 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.487375 (+ 348us) server_negotiation.cc:407] Connection header received
0927 23:00:58.487728 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.487733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.487816 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.487947 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:00:58.488476 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.489466 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.491470 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.492995 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.493212 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.493327 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.493807 (+ 480us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.494008 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260927 23:00:58.494380 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.486485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:43659 (local address 127.4.74.130:57033)
0927 23:00:58.487202 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:58.487219 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:58.487473 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:58.487987 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:58.487990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:58.488018 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:00:58.488305 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.488312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.489628 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.489642 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.491300 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.491313 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.493291 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.493301 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.493544 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.493601 (+ 57us) client_negotiation.cc:769] Sending connection context
0927 23:00:58.493745 (+ 144us) client_negotiation.cc:240] Negotiation successful
0927 23:00:58.493913 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":1,"thread_start_us":348,"threads_started":1}
I20260927 23:00:58.495000 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.485603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.4.74.130:38077)
0927 23:00:58.486379 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0927 23:00:58.486422 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 23:00:58.486690 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:00:58.487268 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:00:58.487271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:00:58.487293 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:00:58.487755 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.487767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.489264 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.489268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.491119 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:00:58.491134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.492815 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.492824 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:00:58.493070 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.493132 (+ 62us) client_negotiation.cc:769] Sending connection context
0927 23:00:58.494238 (+ 1106us) client_negotiation.cc:240] Negotiation successful
0927 23:00:58.494483 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":73,"thread_start_us":392,"threads_started":1}
I20260927 23:00:58.495518 18747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83fb8c3a28546a9a940cfaf3d0768ca" candidate_uuid: "db08bb386fa447cbb7f5a36eaac6b648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab3f836c98184076be1ba35724985bbf" is_pre_election: true
I20260927 23:00:58.496136 18747 raft_consensus.cc:2454] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db08bb386fa447cbb7f5a36eaac6b648 in term 0.
I20260927 23:00:58.496395 18479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83fb8c3a28546a9a940cfaf3d0768ca" candidate_uuid: "db08bb386fa447cbb7f5a36eaac6b648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0eac920703e24a4c8b719e86415da811" is_pre_election: true
I20260927 23:00:58.496935 18546 leader_election.cc:302] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab3f836c98184076be1ba35724985bbf, db08bb386fa447cbb7f5a36eaac6b648; no voters:
I20260927 23:00:58.497032 18479 raft_consensus.cc:2454] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db08bb386fa447cbb7f5a36eaac6b648 in term 0.
I20260927 23:00:58.497413 18818 raft_consensus.cc:2781] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:00:58.497610 18818 raft_consensus.cc:482] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:00:58.497781 18818 raft_consensus.cc:3037] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:58.500797 18818 raft_consensus.cc:504] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } }
I20260927 23:00:58.501605 18818 leader_election.cc:288] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [CANDIDATE]: Term 1 election: Requested vote from peers 0eac920703e24a4c8b719e86415da811 (127.4.74.129:44205), ab3f836c98184076be1ba35724985bbf (127.4.74.131:43659)
I20260927 23:00:58.502300 18479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83fb8c3a28546a9a940cfaf3d0768ca" candidate_uuid: "db08bb386fa447cbb7f5a36eaac6b648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0eac920703e24a4c8b719e86415da811"
I20260927 23:00:58.502445 18747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83fb8c3a28546a9a940cfaf3d0768ca" candidate_uuid: "db08bb386fa447cbb7f5a36eaac6b648" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab3f836c98184076be1ba35724985bbf"
I20260927 23:00:58.502560 18479 raft_consensus.cc:3037] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:58.502758 18747 raft_consensus.cc:3037] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:00:58.505245 18747 raft_consensus.cc:2454] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db08bb386fa447cbb7f5a36eaac6b648 in term 1.
I20260927 23:00:58.505293 18479 raft_consensus.cc:2454] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db08bb386fa447cbb7f5a36eaac6b648 in term 1.
I20260927 23:00:58.505812 18546 leader_election.cc:302] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab3f836c98184076be1ba35724985bbf, db08bb386fa447cbb7f5a36eaac6b648; no voters:
I20260927 23:00:58.506234 18818 raft_consensus.cc:2781] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:00:58.506927 18818 raft_consensus.cc:686] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 1 LEADER]: Becoming Leader. State: Replica: db08bb386fa447cbb7f5a36eaac6b648, State: Running, Role: LEADER
I20260927 23:00:58.507675 18818 consensus_queue.cc:237] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } }
I20260927 23:00:58.514981 18334 catalog_manager.cc:5495] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 reported cstate change: term changed from 0 to 1, leader changed from <none> to db08bb386fa447cbb7f5a36eaac6b648 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "db08bb386fa447cbb7f5a36eaac6b648" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } health_report { overall_health: UNKNOWN } } }
W20260927 23:00:58.517331 18659 tablet.cc:2369] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:00:58.568614 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.561309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.4.74.130:38959)
0927 23:00:58.561551 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.561556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.561578 (+ 22us) server_negotiation.cc:407] Connection header received
0927 23:00:58.561701 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.561705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.561802 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.561992 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:00:58.562548 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.563444 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.565740 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.566303 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.566460 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.567115 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:00:58.567919 (+ 804us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:00:58.568072 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.568228 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.568329 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 23:00:58.579833 18747 raft_consensus.cc:1261] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf [term 1 FOLLOWER]: Refusing update from remote peer db08bb386fa447cbb7f5a36eaac6b648: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:00:58.579833 18479 raft_consensus.cc:1261] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811 [term 1 FOLLOWER]: Refusing update from remote peer db08bb386fa447cbb7f5a36eaac6b648: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:00:58.581137 18822 consensus_queue.cc:1035] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:00:58.581597 18818 consensus_queue.cc:1035] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:00:58.618121 18334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:00:58.618443 18334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:00:58.622574 18334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Sending DeleteTablet for 3 replicas of tablet b83fb8c3a28546a9a940cfaf3d0768ca
I20260927 23:00:58.624159 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 18396
I20260927 23:00:58.624284 18727 tablet_service.cc:1514] Processing DeleteTablet for tablet b83fb8c3a28546a9a940cfaf3d0768ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:00:58 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:34846
I20260927 23:00:58.624508 18459 tablet_service.cc:1514] Processing DeleteTablet for tablet b83fb8c3a28546a9a940cfaf3d0768ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:00:58 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:41262
I20260927 23:00:58.625012 18593 tablet_service.cc:1514] Processing DeleteTablet for tablet b83fb8c3a28546a9a940cfaf3d0768ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:00:58 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:35860
I20260927 23:00:58.634747 18837 tablet_replica.cc:330] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: stopping tablet replica
I20260927 23:00:58.636059 18835 tablet_replica.cc:330] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: stopping tablet replica
I20260927 23:00:58.636420 18837 raft_consensus.cc:2229] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:00:58.636860 18835 raft_consensus.cc:2229] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:00:58.637485 18837 raft_consensus.cc:2258] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:00:58.637549 18835 raft_consensus.cc:2258] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:00:58.638617 18836 tablet_replica.cc:330] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: stopping tablet replica
I20260927 23:00:58.639256 18836 raft_consensus.cc:2229] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:00:58.639971 18836 raft_consensus.cc:2258] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:00:58.640441 18837 ts_tablet_manager.cc:1905] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:00:58.643354 18835 ts_tablet_manager.cc:1905] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:00:58.644512 18836 ts_tablet_manager.cc:1905] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:00:58.648680 18837 ts_tablet_manager.cc:1918] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:00:58.649001 18837 log.cc:1196] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/b83fb8c3a28546a9a940cfaf3d0768ca
I20260927 23:00:58.649693 18837 ts_tablet_manager.cc:1939] T b83fb8c3a28546a9a940cfaf3d0768ca P db08bb386fa447cbb7f5a36eaac6b648: Deleting consensus metadata
I20260927 23:00:58.652069 18836 ts_tablet_manager.cc:1918] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:00:58.652010 18320 catalog_manager.cc:4841] TS db08bb386fa447cbb7f5a36eaac6b648 (127.4.74.130:38959): tablet b83fb8c3a28546a9a940cfaf3d0768ca (table test-table [id=4204e54cd05245bd85866da9c3a7166c]) successfully deleted
I20260927 23:00:58.652661 18835 ts_tablet_manager.cc:1918] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:00:58.652925 18835 log.cc:1196] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/b83fb8c3a28546a9a940cfaf3d0768ca
I20260927 23:00:58.653146 18836 log.cc:1196] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/b83fb8c3a28546a9a940cfaf3d0768ca
I20260927 23:00:58.653716 18835 ts_tablet_manager.cc:1939] T b83fb8c3a28546a9a940cfaf3d0768ca P ab3f836c98184076be1ba35724985bbf: Deleting consensus metadata
I20260927 23:00:58.653825 18836 ts_tablet_manager.cc:1939] T b83fb8c3a28546a9a940cfaf3d0768ca P 0eac920703e24a4c8b719e86415da811: Deleting consensus metadata
I20260927 23:00:58.655972 18319 catalog_manager.cc:4841] TS 0eac920703e24a4c8b719e86415da811 (127.4.74.129:44205): tablet b83fb8c3a28546a9a940cfaf3d0768ca (table test-table [id=4204e54cd05245bd85866da9c3a7166c]) successfully deleted
I20260927 23:00:58.656281 18319 catalog_manager.cc:4841] TS ab3f836c98184076be1ba35724985bbf (127.4.74.131:43659): tablet b83fb8c3a28546a9a940cfaf3d0768ca (table test-table [id=4204e54cd05245bd85866da9c3a7166c]) successfully deleted
Sep 27 23:00:58 dist-test-slave-5pd3 krb5kdc[18290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 23:00:58.667030 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.625970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41268 (local address 127.4.74.129:44205)
0927 23:00:58.626182 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.626191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.635608 (+ 9417us) server_negotiation.cc:407] Connection header received
0927 23:00:58.636829 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.636835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.636956 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.637051 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:58.639683 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.640886 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.652411 (+ 11525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.652974 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.653113 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.658773 (+ 5660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:58.658799 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:58.658839 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:58.658907 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:58.663334 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:58.664407 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:58.664423 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:58.664427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:58.664635 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:58.665065 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:58.665071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:58.665074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:58.665640 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:58.666020 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.666286 (+ 266us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.666570 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 23:00:58.668085 18519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:58.762287 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 18396
I20260927 23:00:58.782248 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 18530
Sep 27 23:00:58 dist-test-slave-5pd3 krb5kdc[18290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 23:00:58.805339 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.783380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.4.74.130:38959)
0927 23:00:58.783689 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.783695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.783711 (+ 16us) server_negotiation.cc:407] Connection header received
0927 23:00:58.783766 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.783770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.783837 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.783960 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:58.785347 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.786258 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.787648 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.788075 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.788216 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.798572 (+ 10356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:58.798591 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:58.798607 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:58.798667 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:58.802176 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:58.803016 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:58.803030 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:58.803032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:58.803182 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:58.803631 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:58.803636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:58.803639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:58.804040 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:58.804458 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.804708 (+ 250us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.804962 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260927 23:00:58.806434 18653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:58.943715 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 18530
I20260927 23:00:58.966478 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 18664
Sep 27 23:00:58 dist-test-slave-5pd3 krb5kdc[18290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 23:00:58.984843 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:00:58.967391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34848 (local address 127.4.74.131:43659)
0927 23:00:58.967651 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0927 23:00:58.967658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:00:58.967680 (+ 22us) server_negotiation.cc:407] Connection header received
0927 23:00:58.967900 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:00:58.967904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:00:58.967976 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:00:58.968092 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:00:58.969833 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.970783 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.972374 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:00:58.972885 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:00:58.973032 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:00:58.977865 (+ 4833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:00:58.977884 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:00:58.977903 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:00:58.977949 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:00:58.981198 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:58.982050 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:58.982057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:58.982060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:58.982272 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:00:58.982726 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:00:58.982731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:00:58.982735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:00:58.983178 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:00:58.983481 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0927 23:00:58.984163 (+ 682us) server_negotiation.cc:299] Negotiation successful
0927 23:00:58.984470 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260927 23:00:58.985934 18787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:59.083742 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 18664
I20260927 23:00:59.103921 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 18303
I20260927 23:00:59.105479 18365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:00:59.221808 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 18303
I20260927 23:00:59.249163 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:43367
--webserver_interface=127.4.74.190
--webserver_port=45127
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:43367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:00:59.612195 18843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:00:59.612588 18843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:00:59.612756 18843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:00:59.620221 18843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:00:59.620421 18843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:00:59.620539 18843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:00:59.620653 18843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:00:59.620728 18843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:00:59.631155 18843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:43367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:43367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=45127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:00:59.632910 18843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:00:59.634693 18843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:00:59.664116 18852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:59.664163 18849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:00:59.664218 18850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:00:59.665864 18843 server_base.cc:1034] running on GCE node
I20260927 23:00:59.667016 18843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:00:59.668910 18843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:00:59.670224 18843 hybrid_clock.cc:648] HybridClock initialized: now 1790550059670163 us; error 57 us; skew 500 ppm
I20260927 23:00:59.670746 18843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:00:59 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550059, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:00:59.674839 18843 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:00:59.676827 18843 webserver.cc:460] Webserver started at http://127.4.74.190:45127/ using document root <none> and password file <none>
I20260927 23:00:59.677604 18843 fs_manager.cc:359] Metadata directory not provided
I20260927 23:00:59.677731 18843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:00:59.684618 18843 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260927 23:00:59.688298 18859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:00:59.690188 18843 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260927 23:00:59.690389 18843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "08b5464f552447d7832812283a3399fd"
format_stamp: "Formatted at 2026-09-27 23:00:56 on dist-test-slave-5pd3"
server_key: "1cce4809e607ffe9d11b0effe846ae91"
server_key_iv: "dda73a9758a91e300a72200af05129db"
server_key_version: "encryptionkey@0"
I20260927 23:00:59.691818 18843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:00:59.710391 18843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:00:59.716132 18843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:00:59.716468 18843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:00:59.740180 18843 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:43367
I20260927 23:00:59.740294 18910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:43367 every 8 connection(s)
I20260927 23:00:59.741995 18843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:00:59.750149 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 18843
I20260927 23:00:59.751067 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:44205
--local_ip_for_outbound_sockets=127.4.74.129
--tserver_master_addrs=127.4.74.190:43367
--webserver_port=33317
--webserver_interface=127.4.74.129
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:00:59.752725 18911 sys_catalog.cc:263] Verifying existing consensus state
I20260927 23:00:59.757165 18911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Bootstrap starting.
I20260927 23:00:59.784097 18911 log.cc:824] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Log is configured to *not* fsync() on all Append() calls
I20260927 23:00:59.800177 18911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 23:00:59.800884 18911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Bootstrap complete.
I20260927 23:00:59.812471 18911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:00:59.813647 18911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08b5464f552447d7832812283a3399fd, State: Initialized, Role: FOLLOWER
I20260927 23:00:59.814471 18911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:00:59.814788 18911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:00:59.814949 18911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:00:59.815157 18911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 1 FOLLOWER]: Advancing to term 2
I20260927 23:00:59.820119 18911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:00:59.820910 18911 leader_election.cc:302] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08b5464f552447d7832812283a3399fd; no voters:
I20260927 23:00:59.821933 18911 leader_election.cc:288] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 23:00:59.822126 18915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 2 FOLLOWER]: Leader election won for term 2
I20260927 23:00:59.823688 18915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 2 LEADER]: Becoming Leader. State: Replica: 08b5464f552447d7832812283a3399fd, State: Running, Role: LEADER
I20260927 23:00:59.824492 18915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:00:59.825179 18911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:00:59.827723 18917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08b5464f552447d7832812283a3399fd. Latest consensus state: current_term: 2 leader_uuid: "08b5464f552447d7832812283a3399fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } } }
I20260927 23:00:59.828264 18917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:59.829077 18916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "08b5464f552447d7832812283a3399fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } } }
I20260927 23:00:59.829391 18916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: This master's current role is: LEADER
I20260927 23:00:59.839427 18920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:00:59.849969 18920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:00:59.852013 18920 catalog_manager.cc:1259] Loaded cluster ID: ca08be00a6014afea7d21840330483fa
I20260927 23:00:59.852272 18920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:00:59.857440 18920 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260927 23:00:59.876777 18920 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260927 23:00:59.889809 18920 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:00:59.892195 18920 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Loaded TSK: 0
I20260927 23:00:59.893954 18920 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 23:01:00.151970 18913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:00.152288 18913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:00.152431 18913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:00.160310 18913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:00.160446 18913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:00.160633 18913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:01:00.171828 18913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:44205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=33317
--tserver_master_addrs=127.4.74.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:00.173316 18913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:00.174762 18913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:00.192039 18941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:00.192246 18939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:00.192447 18938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:00.193828 18913 server_base.cc:1034] running on GCE node
I20260927 23:01:00.194633 18913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:00.195803 18913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:00.199239 18913 hybrid_clock.cc:648] HybridClock initialized: now 1790550060199102 us; error 124 us; skew 500 ppm
I20260927 23:01:00.200062 18913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:00 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550060, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:00.203971 18913 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:01:00.205885 18913 webserver.cc:460] Webserver started at http://127.4.74.129:33317/ using document root <none> and password file <none>
I20260927 23:01:00.206748 18913 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:00.206904 18913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:00.212957 18913 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:01:00.215960 18948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:00.217538 18913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260927 23:01:00.217715 18913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "0eac920703e24a4c8b719e86415da811"
format_stamp: "Formatted at 2026-09-27 23:00:56 on dist-test-slave-5pd3"
server_key: "88a67e1c89beec43edce4c38438b5be5"
server_key_iv: "64e77e4f85576aa97e670a5695425631"
server_key_version: "encryptionkey@0"
I20260927 23:01:00.218747 18913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:00.237051 18913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:00.242017 18913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:00.242337 18913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:00.243539 18913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:00.245216 18913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:00.245316 18913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:00.245434 18913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:00.245543 18913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:00.286798 18913 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:44205
I20260927 23:01:00.286850 19060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:44205 every 8 connection(s)
I20260927 23:01:00.288349 18913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:01:00.289253 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 18913
I20260927 23:01:00.290236 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:38959
--local_ip_for_outbound_sockets=127.4.74.130
--tserver_master_addrs=127.4.74.190:43367
--webserver_port=33519
--webserver_interface=127.4.74.130
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 23:01:00 dist-test-slave-5pd3 krb5kdc[18290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550060, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:01:00.320120 19064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:00.291946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:46225)
0927 23:01:00.293122 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:00.293172 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:00.294995 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:00.297280 (+ 2285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:00.297289 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:00.297962 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:01:00.298779 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:00.298799 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:00.301196 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:00.301201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:00.302611 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:00.302627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:00.304420 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:00.304425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:00.304669 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:00.305557 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:01:00.305581 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:01:00.308360 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:01:00.314539 (+ 6179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:00.314548 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:00.314564 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:00.315146 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:00.315994 (+ 848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:00.315998 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:00.316001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:00.316271 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:00.317693 (+ 1422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:01:00.317700 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:01:00.318124 (+ 424us) client_negotiation.cc:769] Sending connection context
0927 23:01:00.318410 (+ 286us) client_negotiation.cc:240] Negotiation successful
0927 23:01:00.318881 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"thread_start_us":643,"threads_started":1}
I20260927 23:01:00.321254 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:00.291900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:46225 (local address 127.4.74.190:43367)
0927 23:01:00.292710 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:00.292733 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:00.293346 (+ 613us) server_negotiation.cc:407] Connection header received
0927 23:01:00.295816 (+ 2470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:00.295844 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:00.296486 (+ 642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:00.297038 (+ 552us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:00.299071 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:00.300966 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:00.302816 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:00.304124 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:00.304375 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:00.308839 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:00.308855 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:00.308886 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:00.308969 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:00.314191 (+ 5222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:00.315360 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:00.315375 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:00.315389 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:00.315763 (+ 374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:00.316431 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:00.316440 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:00.316445 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:00.317468 (+ 1023us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:00.317745 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:00.318788 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0927 23:01:00.319500 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":303,"threads_started":1}
I20260927 23:01:00.324426 19061 heartbeater.cc:347] Connected to a master server at 127.4.74.190:43367
I20260927 23:01:00.324790 19061 heartbeater.cc:464] Registering TS with master...
I20260927 23:01:00.325604 19061 heartbeater.cc:511] Master 127.4.74.190:43367 requested a full tablet report, sending...
I20260927 23:01:00.328032 18876 ts_manager.cc:194] Registered new tserver with Master: 0eac920703e24a4c8b719e86415da811 (127.4.74.129:44205)
I20260927 23:01:00.329638 18876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:46225
W20260927 23:01:00.680284 19065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:00.680660 19065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:00.680837 19065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:00.688786 19065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:00.688983 19065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:00.689190 19065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:01:00.700381 19065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:38959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=33519
--tserver_master_addrs=127.4.74.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:00.701879 19065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:00.703410 19065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:00.719340 19070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:00.719476 19071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:00.720103 19073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:00.720497 19065 server_base.cc:1034] running on GCE node
I20260927 23:01:00.721225 19065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:00.722398 19065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:00.723735 19065 hybrid_clock.cc:648] HybridClock initialized: now 1790550060723665 us; error 64 us; skew 500 ppm
I20260927 23:01:00.724174 19065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:00 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550060, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:00.727954 19065 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:01:00.729755 19065 webserver.cc:460] Webserver started at http://127.4.74.130:33519/ using document root <none> and password file <none>
I20260927 23:01:00.730545 19065 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:00.730716 19065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:00.737023 19065 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:01:00.740947 19080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:00.743021 19065 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:01:00.743281 19065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "db08bb386fa447cbb7f5a36eaac6b648"
format_stamp: "Formatted at 2026-09-27 23:00:57 on dist-test-slave-5pd3"
server_key: "4782a582dff250eb8edd8b1febcd57ae"
server_key_iv: "533b9ef41b636a2d99c7aa642ed86f2c"
server_key_version: "encryptionkey@0"
I20260927 23:01:00.744781 19065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:00.769963 19065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:00.775411 19065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:00.775758 19065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:00.776904 19065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:00.778476 19065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:00.778589 19065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:00.778741 19065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:00.778839 19065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:00.818236 19065 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:38959
I20260927 23:01:00.818409 19192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:38959 every 8 connection(s)
I20260927 23:01:00.819900 19065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:01:00.826622 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 19065
I20260927 23:01:00.827406 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:43659
--local_ip_for_outbound_sockets=127.4.74.131
--tserver_master_addrs=127.4.74.190:43367
--webserver_port=43657
--webserver_interface=127.4.74.131
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 23:01:00 dist-test-slave-5pd3 krb5kdc[18290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550060, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:01:00.848703 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:00.823276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:56591 (local address 127.4.74.190:43367)
0927 23:01:00.823968 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:00.823976 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:00.825128 (+ 1152us) server_negotiation.cc:407] Connection header received
0927 23:01:00.827308 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:00.827314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:00.827438 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:00.827610 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:00.830554 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:00.831326 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:00.834284 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:00.834893 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:00.835043 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:00.840105 (+ 5062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:00.840115 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:00.840124 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:00.840185 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:00.844354 (+ 4169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:00.845430 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:00.845439 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:00.845446 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:00.845648 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:00.846169 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:00.846176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:00.846182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:00.846592 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:00.846761 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:00.847845 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0927 23:01:00.848180 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":292,"threads_started":1}
I20260927 23:01:00.850059 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:00.823953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:56591)
0927 23:01:00.824899 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:00.824950 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:00.826986 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:00.828109 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:00.828126 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:00.829180 (+ 1054us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:01:00.830255 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:00.830291 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:00.831570 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:00.831578 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:00.834022 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:00.834045 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:00.835259 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:00.835270 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:00.835691 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:00.836688 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:01:00.836738 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:01:00.839788 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:01:00.844585 (+ 4797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:00.844595 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:00.844614 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:00.845238 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:00.845785 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:00.845789 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:00.845791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:00.846022 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:00.846768 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:01:00.846776 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:01:00.847290 (+ 514us) client_negotiation.cc:769] Sending connection context
0927 23:01:00.847700 (+ 410us) client_negotiation.cc:240] Negotiation successful
0927 23:01:00.848262 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":360,"threads_started":1}
I20260927 23:01:00.852322 19193 heartbeater.cc:347] Connected to a master server at 127.4.74.190:43367
I20260927 23:01:00.852733 19193 heartbeater.cc:464] Registering TS with master...
I20260927 23:01:00.853720 19193 heartbeater.cc:511] Master 127.4.74.190:43367 requested a full tablet report, sending...
I20260927 23:01:00.855779 18876 ts_manager.cc:194] Registered new tserver with Master: db08bb386fa447cbb7f5a36eaac6b648 (127.4.74.130:38959)
I20260927 23:01:00.856787 18876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:56591
W20260927 23:01:01.189592 19197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:01.189909 19197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:01.190078 19197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:01.198019 19197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:01.198213 19197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:01.198406 19197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:01:01.208582 19197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:43659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=43657
--tserver_master_addrs=127.4.74.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:01.210001 19197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:01.211355 19197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:01.226910 19203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:01.226926 19202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:01.227057 19205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:01.227842 19197 server_base.cc:1034] running on GCE node
I20260927 23:01:01.228801 19197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:01.230033 19197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:01.231333 19197 hybrid_clock.cc:648] HybridClock initialized: now 1790550061231282 us; error 46 us; skew 500 ppm
I20260927 23:01:01.231792 19197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:01 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550061, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:01.235524 19197 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:01:01.237403 19197 webserver.cc:460] Webserver started at http://127.4.74.131:43657/ using document root <none> and password file <none>
I20260927 23:01:01.238230 19197 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:01.238358 19197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:01.245535 19197 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:01:01.249727 19212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:01.251909 19197 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260927 23:01:01.252197 19197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "ab3f836c98184076be1ba35724985bbf"
format_stamp: "Formatted at 2026-09-27 23:00:58 on dist-test-slave-5pd3"
server_key: "49b12116ee47bc679648ea1fc3e543a9"
server_key_iv: "17c958b34c755be7bc3dfc5d96578f1d"
server_key_version: "encryptionkey@0"
I20260927 23:01:01.253489 19197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:01.273360 19197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:01.279234 19197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:01.279681 19197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:01.281212 19197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:01.283033 19197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:01.283161 19197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:01.283349 19197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:01.283432 19197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:01.326048 19197 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:43659
I20260927 23:01:01.326215 19324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:43659 every 8 connection(s)
I20260927 23:01:01.327993 19197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:01:01.331996 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 19197
I20260927 23:01:01.332959 19061 heartbeater.cc:503] Master 127.4.74.190:43367 was elected leader, sending a full tablet report...
Sep 27 23:01:01 dist-test-slave-5pd3 krb5kdc[18290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550061, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:01:01.355377 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.333921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59904 (local address 127.4.74.190:43367)
0927 23:01:01.337588 (+ 3667us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.337594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.338178 (+ 584us) server_negotiation.cc:407] Connection header received
0927 23:01:01.338418 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.338430 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.338498 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.338621 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:01.340668 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.341523 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.343033 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.343518 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.343701 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.346840 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:01.346846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:01.346850 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:01.346913 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:01.350955 (+ 4042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:01.352071 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:01.352080 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:01.352085 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:01.352409 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:01.353083 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:01.353091 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:01.353098 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:01.353787 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:01.354110 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.354543 (+ 433us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.354863 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":3540,"spinlock_wait_cycles":3584,"thread_start_us":3371,"threads_started":1}
I20260927 23:01:01.355655 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.331297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:41925 (local address 127.4.74.190:43367)
0927 23:01:01.331650 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.331659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.333065 (+ 1406us) server_negotiation.cc:407] Connection header received
0927 23:01:01.337073 (+ 4008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.337082 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.337237 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.337397 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:01.339706 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.340663 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.342279 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.342694 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.342834 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.346774 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:01.346780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:01.346783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:01.346828 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:01.350820 (+ 3992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:01.352073 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:01.352080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:01.352085 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:01.352404 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:01.353116 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:01.353120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:01.353123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:01.353787 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:01.353977 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.354968 (+ 991us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.355211 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":130,"spinlock_wait_cycles":11904}
I20260927 23:01:01.356720 19327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.331734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:41925)
0927 23:01:01.332838 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:01.332897 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:01.336503 (+ 3606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:01.337999 (+ 1496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:01.338013 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:01.338705 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:01:01.339466 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.339486 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.340829 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.340836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.342128 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.342140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.342925 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.342932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.343177 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.343978 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:01:01.344002 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:01:01.346506 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:01:01.350991 (+ 4485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:01.351002 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:01.351024 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:01.351785 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:01.352645 (+ 860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:01.352651 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:01.352654 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:01.352964 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:01.353956 (+ 992us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:01:01.353964 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:01:01.354492 (+ 528us) client_negotiation.cc:769] Sending connection context
0927 23:01:01.354837 (+ 345us) client_negotiation.cc:240] Negotiation successful
0927 23:01:01.355398 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":415,"threads_started":1}
I20260927 23:01:01.358618 19325 heartbeater.cc:347] Connected to a master server at 127.4.74.190:43367
I20260927 23:01:01.359123 19325 heartbeater.cc:464] Registering TS with master...
I20260927 23:01:01.360162 19325 heartbeater.cc:511] Master 127.4.74.190:43367 requested a full tablet report, sending...
I20260927 23:01:01.361932 18875 ts_manager.cc:194] Registered new tserver with Master: ab3f836c98184076be1ba35724985bbf (127.4.74.131:43659)
I20260927 23:01:01.363003 18875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:41925
I20260927 23:01:01.372589 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:01:01.380900 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.373927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.4.74.190:43367)
0927 23:01:01.374545 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.374551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.374571 (+ 20us) server_negotiation.cc:407] Connection header received
0927 23:01:01.374788 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.374792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.374980 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.375137 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:01:01.375762 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.376639 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.378333 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.378865 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.378987 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.379517 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:01:01.380294 (+ 777us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:01:01.380417 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.380565 (+ 148us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.380650 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457}
I20260927 23:01:01.382064 18875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:01:01.390336 18875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:01:01.430850 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.405519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:43659 (local address 127.0.0.1:34852)
0927 23:01:01.416660 (+ 11141us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:01.416746 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:01.417111 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:01.420304 (+ 3193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:01.420317 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:01.420361 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:01.420824 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.420840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.423301 (+ 2461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.423310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.426668 (+ 3358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.426703 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.429523 (+ 2820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.429531 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.429866 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.429961 (+ 95us) client_negotiation.cc:769] Sending connection context
0927 23:01:01.430139 (+ 178us) client_negotiation.cc:240] Negotiation successful
0927 23:01:01.430384 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10613,"thread_start_us":12727,"threads_started":1}
I20260927 23:01:01.430958 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.405890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.4.74.131:43659)
0927 23:01:01.418092 (+ 12202us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.418105 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.418151 (+ 46us) server_negotiation.cc:407] Connection header received
0927 23:01:01.418296 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.418307 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.418686 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.419934 (+ 1248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:01.421048 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.423061 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.426958 (+ 3897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.429239 (+ 2281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.429399 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.429680 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.430253 (+ 573us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.430499 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11992,"thread_start_us":11793,"threads_started":1}
I20260927 23:01:01.432374 19335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.405962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:38959 (local address 127.0.0.1:35894)
0927 23:01:01.418483 (+ 12521us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:01.418506 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:01.418652 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:01.421614 (+ 2962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:01.421625 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:01.421689 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:01.422317 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.422339 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.426387 (+ 4048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.426409 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.429110 (+ 2701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.429132 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.431374 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.431381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.431639 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.431692 (+ 53us) client_negotiation.cc:769] Sending connection context
0927 23:01:01.431766 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 23:01:01.431937 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12389,"thread_start_us":12818,"threads_started":1}
I20260927 23:01:01.432840 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.406395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35894 (local address 127.4.74.130:38959)
0927 23:01:01.420776 (+ 14381us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.420791 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.420824 (+ 33us) server_negotiation.cc:407] Connection header received
0927 23:01:01.420936 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.420947 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.421290 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.421528 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:01.422717 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.426181 (+ 3464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.429364 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.431060 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.431881 (+ 821us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.432131 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.432221 (+ 90us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.432447 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14171,"spinlock_wait_cycles":40192,"thread_start_us":14127,"threads_started":1}
I20260927 23:01:01.436062 19260 tablet_service.cc:1467] Processing CreateTablet for tablet 3852e79a5a534a4c89f78137cb0ff077 (DEFAULT_TABLE table=test-table [id=71adb513639c4fa1b7547bd8a5f2d313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:01:01.437960 19260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3852e79a5a534a4c89f78137cb0ff077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:01:01.438042 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.421999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.0.0.1:41280)
0927 23:01:01.427633 (+ 5634us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:01.427665 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:01.427857 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:01.429123 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:01.429128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:01.429158 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:01.429726 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.429740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.432040 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.432046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.434952 (+ 2906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.434965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.437175 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.437182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.437376 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.437414 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 23:01:01.437520 (+ 106us) client_negotiation.cc:240] Negotiation successful
0927 23:01:01.437682 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5380,"thread_start_us":5158,"threads_started":1}
I20260927 23:01:01.438205 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.420217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.4.74.129:44205)
0927 23:01:01.426341 (+ 6124us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.426351 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.427784 (+ 1433us) server_negotiation.cc:407] Connection header received
0927 23:01:01.428139 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.428162 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.428549 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.428799 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:01.430010 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.431819 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.435146 (+ 3327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.436809 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.437014 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.437278 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.437608 (+ 330us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.437799 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5893,"thread_start_us":5660,"threads_started":1}
I20260927 23:01:01.438133 19128 tablet_service.cc:1467] Processing CreateTablet for tablet 3852e79a5a534a4c89f78137cb0ff077 (DEFAULT_TABLE table=test-table [id=71adb513639c4fa1b7547bd8a5f2d313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:01:01.439860 19128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3852e79a5a534a4c89f78137cb0ff077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:01:01.442708 18996 tablet_service.cc:1467] Processing CreateTablet for tablet 3852e79a5a534a4c89f78137cb0ff077 (DEFAULT_TABLE table=test-table [id=71adb513639c4fa1b7547bd8a5f2d313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:01:01.444490 18996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3852e79a5a534a4c89f78137cb0ff077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:01:01.454241 19341 tablet_bootstrap.cc:492] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: Bootstrap starting.
I20260927 23:01:01.459846 19341 tablet_bootstrap.cc:654] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: Neither blocks nor log segments found. Creating new log.
I20260927 23:01:01.461532 19341 log.cc:824] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:01.462740 19342 tablet_bootstrap.cc:492] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: Bootstrap starting.
I20260927 23:01:01.465845 19341 tablet_bootstrap.cc:492] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: No bootstrap required, opened a new log
I20260927 23:01:01.466077 19344 tablet_bootstrap.cc:492] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Bootstrap starting.
I20260927 23:01:01.466902 19341 ts_tablet_manager.cc:1397] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260927 23:01:01.467615 19342 tablet_bootstrap.cc:654] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: Neither blocks nor log segments found. Creating new log.
I20260927 23:01:01.468922 19342 log.cc:824] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:01.471127 19344 tablet_bootstrap.cc:654] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Neither blocks nor log segments found. Creating new log.
I20260927 23:01:01.472638 19344 log.cc:824] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:01.476177 19344 tablet_bootstrap.cc:492] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: No bootstrap required, opened a new log
I20260927 23:01:01.476459 19342 tablet_bootstrap.cc:492] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: No bootstrap required, opened a new log
I20260927 23:01:01.476583 19344 ts_tablet_manager.cc:1397] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260927 23:01:01.476815 19342 ts_tablet_manager.cc:1397] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260927 23:01:01.478713 19341 raft_consensus.cc:348] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } }
I20260927 23:01:01.479321 19341 raft_consensus.cc:374] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:01:01.479446 19341 raft_consensus.cc:729] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db08bb386fa447cbb7f5a36eaac6b648, State: Initialized, Role: FOLLOWER
I20260927 23:01:01.480281 19341 consensus_queue.cc:260] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } }
I20260927 23:01:01.481832 19193 heartbeater.cc:503] Master 127.4.74.190:43367 was elected leader, sending a full tablet report...
I20260927 23:01:01.482623 19341 ts_tablet_manager.cc:1428] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260927 23:01:01.487154 19344 raft_consensus.cc:348] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } }
I20260927 23:01:01.487730 19344 raft_consensus.cc:374] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:01:01.487851 19344 raft_consensus.cc:729] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eac920703e24a4c8b719e86415da811, State: Initialized, Role: FOLLOWER
I20260927 23:01:01.488688 19344 consensus_queue.cc:260] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } }
I20260927 23:01:01.488662 19342 raft_consensus.cc:348] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } }
I20260927 23:01:01.489377 19342 raft_consensus.cc:374] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:01:01.489519 19342 raft_consensus.cc:729] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab3f836c98184076be1ba35724985bbf, State: Initialized, Role: FOLLOWER
I20260927 23:01:01.490527 19342 consensus_queue.cc:260] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } }
I20260927 23:01:01.491341 19344 ts_tablet_manager.cc:1428] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260927 23:01:01.492013 19325 heartbeater.cc:503] Master 127.4.74.190:43367 was elected leader, sending a full tablet report...
I20260927 23:01:01.493018 19342 ts_tablet_manager.cc:1428] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
W20260927 23:01:01.541653 19062 tablet.cc:2369] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 23:01:01.572382 19194 tablet.cc:2369] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 23:01:01.580067 19326 tablet.cc:2369] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:01:01.656257 19349 raft_consensus.cc:482] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:01:01.656636 19349 raft_consensus.cc:504] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } }
I20260927 23:01:01.658631 19349 leader_election.cc:288] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0eac920703e24a4c8b719e86415da811 (127.4.74.129:44205), db08bb386fa447cbb7f5a36eaac6b648 (127.4.74.130:38959)
I20260927 23:01:01.665426 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.659256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:57833 (local address 127.4.74.129:44205)
0927 23:01:01.659533 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.659602 (+ 69us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.659639 (+ 37us) server_negotiation.cc:407] Connection header received
0927 23:01:01.659766 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.659772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.659877 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.660002 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:01.660546 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.661318 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.663389 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.664496 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.664680 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.664757 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.665028 (+ 271us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.665157 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260927 23:01:01.665457 19327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.658880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.4.74.131:57833)
0927 23:01:01.659140 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:01.659162 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:01.659310 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:01.660070 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:01.660075 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:01.660109 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:01.660343 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.660357 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.661484 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.661490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.663217 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.663232 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.664666 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.664671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.664863 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.664888 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 23:01:01.664999 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 23:01:01.665149 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":57}
I20260927 23:01:01.666718 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.659226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:38959 (local address 127.4.74.131:44545)
0927 23:01:01.660074 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:01.660092 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:01.660261 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:01.660742 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:01.660748 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:01.660781 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:01.661145 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.661155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.662286 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.662293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.664384 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:01.664399 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.665859 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.665864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:01.666063 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.666102 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 23:01:01.666204 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 23:01:01.666348 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":34,"thread_start_us":477,"threads_started":1}
I20260927 23:01:01.666787 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.659302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:44545 (local address 127.4.74.130:38959)
0927 23:01:01.659528 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.659533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.660173 (+ 640us) server_negotiation.cc:407] Connection header received
0927 23:01:01.660484 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.660491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.660587 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.660706 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:01.661321 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.662109 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.664561 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.665535 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.665727 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.665805 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.666285 (+ 480us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.666501 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 23:01:01.666927 19016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3852e79a5a534a4c89f78137cb0ff077" candidate_uuid: "ab3f836c98184076be1ba35724985bbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0eac920703e24a4c8b719e86415da811" is_pre_election: true
I20260927 23:01:01.667778 19016 raft_consensus.cc:2454] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab3f836c98184076be1ba35724985bbf in term 0.
I20260927 23:01:01.668118 19148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3852e79a5a534a4c89f78137cb0ff077" candidate_uuid: "ab3f836c98184076be1ba35724985bbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db08bb386fa447cbb7f5a36eaac6b648" is_pre_election: true
I20260927 23:01:01.668745 19148 raft_consensus.cc:2454] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab3f836c98184076be1ba35724985bbf in term 0.
I20260927 23:01:01.668761 19213 leader_election.cc:302] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0eac920703e24a4c8b719e86415da811, ab3f836c98184076be1ba35724985bbf; no voters:
I20260927 23:01:01.669312 19349 raft_consensus.cc:2781] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:01:01.669487 19349 raft_consensus.cc:482] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:01:01.669642 19349 raft_consensus.cc:3037] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:01:01.672380 19349 raft_consensus.cc:504] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } }
I20260927 23:01:01.673231 19349 leader_election.cc:288] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [CANDIDATE]: Term 1 election: Requested vote from peers 0eac920703e24a4c8b719e86415da811 (127.4.74.129:44205), db08bb386fa447cbb7f5a36eaac6b648 (127.4.74.130:38959)
I20260927 23:01:01.673938 19016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3852e79a5a534a4c89f78137cb0ff077" candidate_uuid: "ab3f836c98184076be1ba35724985bbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0eac920703e24a4c8b719e86415da811"
I20260927 23:01:01.674141 19148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3852e79a5a534a4c89f78137cb0ff077" candidate_uuid: "ab3f836c98184076be1ba35724985bbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db08bb386fa447cbb7f5a36eaac6b648"
I20260927 23:01:01.674263 19016 raft_consensus.cc:3037] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:01:01.674454 19148 raft_consensus.cc:3037] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:01:01.677016 19148 raft_consensus.cc:2454] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab3f836c98184076be1ba35724985bbf in term 1.
I20260927 23:01:01.677361 19016 raft_consensus.cc:2454] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab3f836c98184076be1ba35724985bbf in term 1.
I20260927 23:01:01.677691 19214 leader_election.cc:302] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab3f836c98184076be1ba35724985bbf, db08bb386fa447cbb7f5a36eaac6b648; no voters:
I20260927 23:01:01.678146 19349 raft_consensus.cc:2781] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:01:01.678723 19349 raft_consensus.cc:686] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 1 LEADER]: Becoming Leader. State: Replica: ab3f836c98184076be1ba35724985bbf, State: Running, Role: LEADER
I20260927 23:01:01.679361 19349 consensus_queue.cc:237] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } } peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } }
I20260927 23:01:01.685581 18876 catalog_manager.cc:5495] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf reported cstate change: term changed from 0 to 1, leader changed from <none> to ab3f836c98184076be1ba35724985bbf (127.4.74.131). New cstate: current_term: 1 leader_uuid: "ab3f836c98184076be1ba35724985bbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab3f836c98184076be1ba35724985bbf" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 } health_report { overall_health: UNKNOWN } } }
I20260927 23:01:01.738062 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.731818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34858 (local address 127.4.74.131:43659)
0927 23:01:01.732071 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.732080 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.732107 (+ 27us) server_negotiation.cc:407] Connection header received
0927 23:01:01.732245 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.732249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.732341 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.732521 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:01:01.732950 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.733830 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.735407 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.735887 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.736034 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.736658 (+ 624us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:01:01.737339 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:01:01.737539 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.737608 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.737749 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260927 23:01:01.749176 19148 raft_consensus.cc:1261] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648 [term 1 FOLLOWER]: Refusing update from remote peer ab3f836c98184076be1ba35724985bbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:01:01.750766 19016 raft_consensus.cc:1261] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811 [term 1 FOLLOWER]: Refusing update from remote peer ab3f836c98184076be1ba35724985bbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:01:01.750628 19351 consensus_queue.cc:1035] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "db08bb386fa447cbb7f5a36eaac6b648" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:01:01.751928 19351 consensus_queue.cc:1035] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0eac920703e24a4c8b719e86415da811" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:01:01.784188 18876 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:01:01.784595 18876 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:01:01.789057 18876 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Sending DeleteTablet for 3 replicas of tablet 3852e79a5a534a4c89f78137cb0ff077
I20260927 23:01:01.790715 18996 tablet_service.cc:1514] Processing DeleteTablet for tablet 3852e79a5a534a4c89f78137cb0ff077 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:01:01 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:41280
I20260927 23:01:01.790778 19128 tablet_service.cc:1514] Processing DeleteTablet for tablet 3852e79a5a534a4c89f78137cb0ff077 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:01:01 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:35894
I20260927 23:01:01.791024 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 18913
I20260927 23:01:01.792017 19260 tablet_service.cc:1514] Processing DeleteTablet for tablet 3852e79a5a534a4c89f78137cb0ff077 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:01:01 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:34852
I20260927 23:01:01.801643 19366 tablet_replica.cc:330] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: stopping tablet replica
I20260927 23:01:01.802289 19366 raft_consensus.cc:2229] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:01:01.803180 19366 raft_consensus.cc:2258] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:01:01.806641 19366 ts_tablet_manager.cc:1905] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:01:01.812325 19364 tablet_replica.cc:330] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: stopping tablet replica
I20260927 23:01:01.812966 19364 raft_consensus.cc:2229] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:01:01.813673 19364 raft_consensus.cc:2258] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:01:01.814721 19365 tablet_replica.cc:330] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: stopping tablet replica
I20260927 23:01:01.815264 19365 raft_consensus.cc:2229] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:01:01.815955 19365 raft_consensus.cc:2258] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:01:01.818886 19365 ts_tablet_manager.cc:1905] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:01:01.821249 19366 ts_tablet_manager.cc:1918] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:01:01.821480 19366 log.cc:1196] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/3852e79a5a534a4c89f78137cb0ff077
I20260927 23:01:01.821856 19364 ts_tablet_manager.cc:1905] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:01:01.822089 19366 ts_tablet_manager.cc:1939] T 3852e79a5a534a4c89f78137cb0ff077 P ab3f836c98184076be1ba35724985bbf: Deleting consensus metadata
I20260927 23:01:01.824299 18860 catalog_manager.cc:4841] TS ab3f836c98184076be1ba35724985bbf (127.4.74.131:43659): tablet 3852e79a5a534a4c89f78137cb0ff077 (table test-table [id=71adb513639c4fa1b7547bd8a5f2d313]) successfully deleted
I20260927 23:01:01.828420 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.795187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41292 (local address 127.4.74.129:44205)
0927 23:01:01.795446 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.795455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.795482 (+ 27us) server_negotiation.cc:407] Connection header received
0927 23:01:01.795599 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.795609 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.795759 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.795905 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:01.798569 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.800369 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.802466 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.803202 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.803350 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.811673 (+ 8323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:01.811706 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:01.811740 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:01.811812 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:01.816498 (+ 4686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:01.819742 (+ 3244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:01.819755 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:01.819759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:01.820029 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:01.824703 (+ 4674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:01.824711 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:01.824716 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:01.825433 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:01.827205 (+ 1772us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.827397 (+ 192us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.827869 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260927 23:01:01.829409 19056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:01.836210 19365 ts_tablet_manager.cc:1918] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:01:01.836431 19365 log.cc:1196] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/3852e79a5a534a4c89f78137cb0ff077
I20260927 23:01:01.837031 19365 ts_tablet_manager.cc:1939] T 3852e79a5a534a4c89f78137cb0ff077 P db08bb386fa447cbb7f5a36eaac6b648: Deleting consensus metadata
I20260927 23:01:01.839291 18861 catalog_manager.cc:4841] TS db08bb386fa447cbb7f5a36eaac6b648 (127.4.74.130:38959): tablet 3852e79a5a534a4c89f78137cb0ff077 (table test-table [id=71adb513639c4fa1b7547bd8a5f2d313]) successfully deleted
I20260927 23:01:01.942948 19364 ts_tablet_manager.cc:1918] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:01:01.943321 19364 log.cc:1196] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/3852e79a5a534a4c89f78137cb0ff077
I20260927 23:01:01.943847 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 18913
W20260927 23:01:01.966177 18860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.74.129:44205 (error 108)
I20260927 23:01:01.966465 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 19065
I20260927 23:01:01.969609 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.968740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.0.0.1:41308)
0927 23:01:01.969236 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:01.969394 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"threads_started":1}
W20260927 23:01:01.970227 18860 catalog_manager.cc:4584] TS 0eac920703e24a4c8b719e86415da811 (127.4.74.129:44205): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3852e79a5a534a4c89f78137cb0ff077: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44205: connect: Connection refused (error 111)
I20260927 23:01:01.985608 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:01.967456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35902 (local address 127.4.74.130:38959)
0927 23:01:01.967722 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:01.967729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:01.967779 (+ 50us) server_negotiation.cc:407] Connection header received
0927 23:01:01.968038 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:01.968045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:01.968192 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:01.968312 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:01.970949 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.971897 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.973964 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:01.974420 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:01.974532 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:01.978282 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:01.978300 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:01.978320 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:01.978370 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:01.981995 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:01.983119 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:01.983130 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:01.983135 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:01.983357 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:01.983867 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:01.983870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:01.983872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:01.984352 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:01.984619 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:01.985026 (+ 407us) server_negotiation.cc:299] Negotiation successful
0927 23:01:01.985241 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 23:01:01.986469 19188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:02.025290 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:02.024018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.0.0.1:41322)
0927 23:01:02.024881 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:02.025066 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":649}
I20260927 23:01:02.026824 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:02.026368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.0.0.1:41328)
0927 23:01:02.026528 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:02.026650 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 23:01:02.088943 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:02.088216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.0.0.1:41334)
0927 23:01:02.088561 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:02.088686 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260927 23:01:02.089308 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 19065
I20260927 23:01:02.097852 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:02.097132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.0.0.1:41336)
0927 23:01:02.097458 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:02.097614 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260927 23:01:02.113476 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 19197
I20260927 23:01:02.131529 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:02.114527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34860 (local address 127.4.74.131:43659)
0927 23:01:02.114733 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:02.114741 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:02.114769 (+ 28us) server_negotiation.cc:407] Connection header received
0927 23:01:02.115029 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:02.115035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:02.115127 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:02.115265 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:02.116817 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:02.117740 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:02.119192 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:02.119776 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:02.119926 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:02.122957 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:02.122978 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:02.122999 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:02.123038 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:02.127341 (+ 4303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:02.128310 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:02.128323 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:02.128329 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:02.128522 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:02.129335 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:02.129344 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:02.129348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:02.130041 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:02.130409 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:02.130759 (+ 350us) server_negotiation.cc:299] Negotiation successful
0927 23:01:02.131086 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 23:01:02.132586 19320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:02.179075 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:02.178314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.0.0.1:41346)
0927 23:01:02.178691 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:02.178866 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260927 23:01:02.180549 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:02.179989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.0.0.1:41350)
0927 23:01:02.180273 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:02.180372 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260927 23:01:02.267032 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 19197
I20260927 23:01:02.287878 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 18843
I20260927 23:01:02.289273 18906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:02.415125 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:02.414114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44205 (local address 127.0.0.1:41356)
0927 23:01:02.414613 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:02.414779 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260927 23:01:02.415423 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 18843
I20260927 23:01:02.439495 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:43367
--webserver_interface=127.4.74.190
--webserver_port=45127
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:43367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:01:02.846836 19373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:02.847213 19373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:02.847363 19373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:02.855924 19373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:01:02.856108 19373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:02.856187 19373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:02.856293 19373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:01:02.856356 19373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:01:02.868147 19373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:43367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:43367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=45127
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:02.869949 19373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:02.871634 19373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:02.887212 19378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:02.887212 19381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:02.887233 19379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:02.888463 19373 server_base.cc:1034] running on GCE node
I20260927 23:01:02.889535 19373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:02.891356 19373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:02.892645 19373 hybrid_clock.cc:648] HybridClock initialized: now 1790550062892557 us; error 82 us; skew 500 ppm
I20260927 23:01:02.893101 19373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:02 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550062, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:02.896714 19373 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:01:02.898598 19373 webserver.cc:460] Webserver started at http://127.4.74.190:45127/ using document root <none> and password file <none>
I20260927 23:01:02.899523 19373 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:02.899731 19373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:02.906168 19373 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260927 23:01:02.912315 19388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:02.914328 19373 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260927 23:01:02.914695 19373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "08b5464f552447d7832812283a3399fd"
format_stamp: "Formatted at 2026-09-27 23:00:56 on dist-test-slave-5pd3"
server_key: "1cce4809e607ffe9d11b0effe846ae91"
server_key_iv: "dda73a9758a91e300a72200af05129db"
server_key_version: "encryptionkey@0"
I20260927 23:01:02.915958 19373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:02.949859 19373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:02.954600 19373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:02.954908 19373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:02.978416 19373 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:43367
I20260927 23:01:02.978458 19439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:43367 every 8 connection(s)
I20260927 23:01:02.980250 19373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:01:02.989677 19440 sys_catalog.cc:263] Verifying existing consensus state
I20260927 23:01:02.989995 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 19373
I20260927 23:01:02.990972 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:44205
--local_ip_for_outbound_sockets=127.4.74.129
--tserver_master_addrs=127.4.74.190:43367
--webserver_port=33317
--webserver_interface=127.4.74.129
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:01:02.994048 19440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Bootstrap starting.
I20260927 23:01:03.021029 19440 log.cc:824] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:03.040994 19440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 23:01:03.041706 19440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd: Bootstrap complete.
I20260927 23:01:03.052618 19440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:01:03.053699 19440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08b5464f552447d7832812283a3399fd, State: Initialized, Role: FOLLOWER
I20260927 23:01:03.054526 19440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:01:03.054807 19440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:01:03.054952 19440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:01:03.055157 19440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 2 FOLLOWER]: Advancing to term 3
I20260927 23:01:03.059962 19440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:01:03.060613 19440 leader_election.cc:302] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08b5464f552447d7832812283a3399fd; no voters:
I20260927 23:01:03.061398 19440 leader_election.cc:288] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 23:01:03.061563 19444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 3 FOLLOWER]: Leader election won for term 3
I20260927 23:01:03.063001 19444 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [term 3 LEADER]: Becoming Leader. State: Replica: 08b5464f552447d7832812283a3399fd, State: Running, Role: LEADER
I20260927 23:01:03.063725 19444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } }
I20260927 23:01:03.064143 19440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:01:03.066684 19446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08b5464f552447d7832812283a3399fd. Latest consensus state: current_term: 3 leader_uuid: "08b5464f552447d7832812283a3399fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } } }
I20260927 23:01:03.066895 19445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "08b5464f552447d7832812283a3399fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b5464f552447d7832812283a3399fd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 43367 } } }
I20260927 23:01:03.067245 19446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: This master's current role is: LEADER
I20260927 23:01:03.067319 19445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b5464f552447d7832812283a3399fd [sys.catalog]: This master's current role is: LEADER
I20260927 23:01:03.081095 19451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:01:03.090068 19451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:01:03.091841 19451 catalog_manager.cc:1259] Loaded cluster ID: ca08be00a6014afea7d21840330483fa
I20260927 23:01:03.091972 19451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260927 23:01:03.093710 19451 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc26d5c29ad google::LogMessage::Fail() at ??:0
@ 0x7fc26d5c588a google::LogMessage::SendToLog() at ??:0
@ 0x7fc26d5c24de google::LogMessage::Flush() at ??:0
@ 0x7fc26d5c3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc282ff5512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc282ff2976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc28307a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc28307a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc27020e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc27021191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc270211799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc2701eceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc2734c66db start_thread at ??:0
@ 0x7fc26c45c61f clone at ??:0
W20260927 23:01:03.466564 19442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:03.466877 19442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:03.466984 19442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:03.474572 19442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:03.474696 19442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:03.474819 19442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:01:03.485111 19442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:44205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=33317
--tserver_master_addrs=127.4.74.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:03.486405 19442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:03.487757 19442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:03.502470 19482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:03.502411 19481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:03.502365 19484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:03.503999 19442 server_base.cc:1034] running on GCE node
I20260927 23:01:03.504769 19442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:03.505934 19442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:03.507243 19442 hybrid_clock.cc:648] HybridClock initialized: now 1790550063507182 us; error 56 us; skew 500 ppm
I20260927 23:01:03.507750 19442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:03 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550063, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:03.511801 19442 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:01:03.513718 19442 webserver.cc:460] Webserver started at http://127.4.74.129:33317/ using document root <none> and password file <none>
I20260927 23:01:03.514519 19442 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:03.514675 19442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:03.521291 19442 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:01:03.525137 19491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:03.526889 19442 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:01:03.527146 19442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "0eac920703e24a4c8b719e86415da811"
format_stamp: "Formatted at 2026-09-27 23:00:56 on dist-test-slave-5pd3"
server_key: "88a67e1c89beec43edce4c38438b5be5"
server_key_iv: "64e77e4f85576aa97e670a5695425631"
server_key_version: "encryptionkey@0"
I20260927 23:01:03.528462 19442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:03.542255 19442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:03.547112 19442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:03.547421 19442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:03.548612 19442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:03.550625 19498 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260927 23:01:03.557482 19498 ts_tablet_manager.cc:1852] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260927 23:01:03.558771 19498 ts_tablet_manager.cc:1905] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:01:03.568182 19498 ts_tablet_manager.cc:1918] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:01:03.568560 19498 ts_tablet_manager.cc:1939] T 3852e79a5a534a4c89f78137cb0ff077 P 0eac920703e24a4c8b719e86415da811: Deleting consensus metadata
I20260927 23:01:03.569124 19442 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260927 23:01:03.569373 19442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.019s user 0.001s sys 0.000s
I20260927 23:01:03.569600 19442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:03.569705 19442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:03.611017 19442 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:44205
I20260927 23:01:03.611073 19604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:44205 every 8 connection(s)
I20260927 23:01:03.612459 19442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:01:03.619128 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:03.616756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:34513)
0927 23:01:03.617817 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:03.618106 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":432,"threads_started":1}
W20260927 23:01:03.620131 19605 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:43367 (0 consecutive failures): Network error: Failed to ping master at 127.4.74.190:43367: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
I20260927 23:01:03.621423 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:03.620949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:48283)
0927 23:01:03.621172 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:03.621274 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260927 23:01:03.622534 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:03.622133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:53757)
0927 23:01:03.622291 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:03.622386 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260927 23:01:03.622996 19605 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 23:01:03.623005 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 19442
I20260927 23:01:03.623801 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:38959
--local_ip_for_outbound_sockets=127.4.74.130
--tserver_master_addrs=127.4.74.190:43367
--webserver_port=33519
--webserver_interface=127.4.74.130
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:01:03.974018 19608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:03.974349 19608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:03.974495 19608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:03.982182 19608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:03.982335 19608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:03.982816 19608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:01:03.993330 19608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:38959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=33519
--tserver_master_addrs=127.4.74.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:03.994761 19608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:03.996269 19608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:04.013170 19614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:04.013244 19613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:04.013250 19616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:04.015754 19608 server_base.cc:1034] running on GCE node
I20260927 23:01:04.016749 19608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:04.018333 19608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:04.019697 19608 hybrid_clock.cc:648] HybridClock initialized: now 1790550064019628 us; error 63 us; skew 500 ppm
I20260927 23:01:04.020287 19608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:04 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550064, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:04.032609 19608 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:01:04.034746 19608 webserver.cc:460] Webserver started at http://127.4.74.130:33519/ using document root <none> and password file <none>
I20260927 23:01:04.035797 19608 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:04.035979 19608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:04.042858 19608 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260927 23:01:04.046550 19623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:04.048628 19608 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260927 23:01:04.048871 19608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "db08bb386fa447cbb7f5a36eaac6b648"
format_stamp: "Formatted at 2026-09-27 23:00:57 on dist-test-slave-5pd3"
server_key: "4782a582dff250eb8edd8b1febcd57ae"
server_key_iv: "533b9ef41b636a2d99c7aa642ed86f2c"
server_key_version: "encryptionkey@0"
I20260927 23:01:04.050138 19608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:04.074879 19608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:04.080312 19608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:04.080647 19608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:04.081923 19608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:04.083832 19608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:04.083961 19608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:04.084108 19608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:04.084210 19608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:04.131637 19608 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:38959
I20260927 23:01:04.131779 19735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:38959 every 8 connection(s)
I20260927 23:01:04.133565 19608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:01:04.139143 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:04.136817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:56075)
0927 23:01:04.137751 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:04.138029 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":379,"threads_started":1}
W20260927 23:01:04.140122 19736 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:43367 (0 consecutive failures): Network error: Failed to ping master at 127.4.74.190:43367: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
I20260927 23:01:04.141070 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:04.140710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:60313)
0927 23:01:04.140885 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:04.140966 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260927 23:01:04.142117 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:04.141780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:37291)
0927 23:01:04.141917 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:04.142007 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260927 23:01:04.142426 19736 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 23:01:04.143971 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 19608
I20260927 23:01:04.144635 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:43659
--local_ip_for_outbound_sockets=127.4.74.131
--tserver_master_addrs=127.4.74.190:43367
--webserver_port=43657
--webserver_interface=127.4.74.131
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:01:04.483204 19739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:04.483518 19739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:04.483641 19739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:04.491079 19739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:04.491214 19739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:04.491341 19739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:01:04.501681 19739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:43659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=43657
--tserver_master_addrs=127.4.74.190:43367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:04.502977 19739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:04.504321 19739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:04.518916 19745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:04.519037 19747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:04.519017 19744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:04.520231 19739 server_base.cc:1034] running on GCE node
I20260927 23:01:04.520951 19739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:04.521984 19739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:04.523250 19739 hybrid_clock.cc:648] HybridClock initialized: now 1790550064523201 us; error 70 us; skew 500 ppm
I20260927 23:01:04.523689 19739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:04 dist-test-slave-5pd3 krb5kdc[18290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550064, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:04.527240 19739 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:01:04.529104 19739 webserver.cc:460] Webserver started at http://127.4.74.131:43657/ using document root <none> and password file <none>
I20260927 23:01:04.529887 19739 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:04.530053 19739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:04.536722 19739 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260927 23:01:04.540086 19754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:04.541774 19739 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260927 23:01:04.541991 19739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "ab3f836c98184076be1ba35724985bbf"
format_stamp: "Formatted at 2026-09-27 23:00:58 on dist-test-slave-5pd3"
server_key: "49b12116ee47bc679648ea1fc3e543a9"
server_key_iv: "17c958b34c755be7bc3dfc5d96578f1d"
server_key_version: "encryptionkey@0"
I20260927 23:01:04.543036 19739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:04.560868 19739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:04.565198 19739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:04.565490 19739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:04.566618 19739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:04.568145 19739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:04.568250 19739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:04.568410 19739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:04.568513 19739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:04.607250 19739 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:43659
I20260927 23:01:04.607425 19866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:43659 every 8 connection(s)
I20260927 23:01:04.608940 19739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:01:04.611063 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 19739
I20260927 23:01:04.620646 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:04.612786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:55307)
0927 23:01:04.619086 (+ 6300us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:04.619375 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5939,"thread_start_us":5716,"threads_started":1}
W20260927 23:01:04.621580 19867 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:43367 (0 consecutive failures): Network error: Failed to ping master at 127.4.74.190:43367: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
I20260927 23:01:04.622864 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:04.622370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:36655)
0927 23:01:04.622604 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:04.622714 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260927 23:01:04.624183 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:04.623796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:42997)
0927 23:01:04.623957 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:04.624056 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260927 23:01:04.624565 19867 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 23:01:04.624774 19871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:04.623755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:56247)
0927 23:01:04.624372 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:04.624509 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":325,"threads_started":1}
I20260927 23:01:05.144326 19872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:05.143130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:50897)
0927 23:01:05.143877 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:05.144050 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":344,"threads_started":1}
I20260927 23:01:05.626364 19873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:05.625373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:33983)
0927 23:01:05.625983 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:05.626104 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":326,"threads_started":1}
I20260927 23:01:05.626844 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:05.625892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:51237)
0927 23:01:05.626497 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:05.626630 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":366,"threads_started":1}
I20260927 23:01:06.146445 19875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:06.145490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:45673)
0927 23:01:06.146088 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:06.146258 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":312,"threads_started":1}
I20260927 23:01:06.628580 19876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:06.627672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:39515)
0927 23:01:06.628302 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:06.628395 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":326,"threads_started":1}
I20260927 23:01:06.629534 19877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:06.628051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:48445)
0927 23:01:06.629111 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:06.629289 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":798,"threads_started":1}
I20260927 23:01:07.148658 19878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:07.147776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:33397)
0927 23:01:07.148345 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:07.148451 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":289,"threads_started":1}
I20260927 23:01:07.631007 19879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:07.629792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:60409)
0927 23:01:07.630563 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:07.630722 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":431,"threads_started":1}
I20260927 23:01:07.632085 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:07.630835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:38821)
0927 23:01:07.631656 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:07.631763 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":523,"threads_started":1}
I20260927 23:01:08.151315 19881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:08.150199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:42983)
0927 23:01:08.150946 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:08.151086 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":340,"threads_started":1}
I20260927 23:01:08.633374 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:08.632283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:37805)
0927 23:01:08.632909 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:08.633035 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":325,"threads_started":1}
I20260927 23:01:08.634263 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:08.633228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:43747)
0927 23:01:08.634004 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:08.634098 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":501,"threads_started":1}
I20260927 23:01:09.153573 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:09.152646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:48849)
0927 23:01:09.153279 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:09.153385 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":338,"threads_started":1}
I20260927 23:01:09.636515 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:09.634744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:41487)
0927 23:01:09.636178 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:09.636291 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1291,"thread_start_us":1188,"threads_started":1}
I20260927 23:01:09.637063 19886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:09.635438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:47809)
0927 23:01:09.636720 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:09.636848 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1143,"thread_start_us":973,"threads_started":1}
I20260927 23:01:10.156039 19887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:10.154816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:39745)
0927 23:01:10.155597 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:10.155740 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":432,"threads_started":1}
I20260927 23:01:10.638697 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:10.637776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:58569)
0927 23:01:10.638401 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:10.638511 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":347,"threads_started":1}
I20260927 23:01:10.639428 19889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:10.638295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:38489)
0927 23:01:10.639017 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:10.639152 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":385,"threads_started":1}
I20260927 23:01:11.158836 19890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:11.157506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:46141)
0927 23:01:11.158408 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:11.158537 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":381,"threads_started":1}
I20260927 23:01:11.641084 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:11.640040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:56657)
0927 23:01:11.640718 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:11.640818 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":331,"threads_started":1}
I20260927 23:01:11.641426 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:11.640492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:58395)
0927 23:01:11.641072 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:11.641165 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":343,"threads_started":1}
I20260927 23:01:12.161420 19893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:12.160333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:45797)
0927 23:01:12.161046 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:12.161169 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":352,"threads_started":1}
I20260927 23:01:12.643699 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:12.642575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:35793)
0927 23:01:12.643234 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:12.643342 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":333,"threads_started":1}
I20260927 23:01:12.643910 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:12.642607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:49555)
0927 23:01:12.643483 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:12.643636 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":406,"threads_started":1}
I20260927 23:01:13.164000 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:13.162914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:43085)
0927 23:01:13.163679 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:13.163793 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":331,"threads_started":1}
I20260927 23:01:13.646306 19898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:13.645104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:45525)
0927 23:01:13.645848 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:13.646014 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":391,"threads_started":1}
I20260927 23:01:13.646515 19897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:13.645071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:48299)
0927 23:01:13.646127 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:13.646234 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":383,"threads_started":1}
I20260927 23:01:14.166667 19899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:14.165244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:47943)
0927 23:01:14.166388 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:14.166488 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1031,"thread_start_us":902,"threads_started":1}
I20260927 23:01:14.649015 19901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:14.647915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:37995)
0927 23:01:14.648691 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:14.648823 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":380,"threads_started":1}
I20260927 23:01:14.649214 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:14.647952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:56517)
0927 23:01:14.648871 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:14.648971 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":664,"threads_started":1}
I20260927 23:01:15.168973 19902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:15.168003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:57555)
0927 23:01:15.168597 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:15.168735 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":277,"threads_started":1}
I20260927 23:01:15.651057 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:15.650109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:57395)
0927 23:01:15.650723 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:15.650837 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":315,"threads_started":1}
I20260927 23:01:15.651433 19904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:15.650292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:33621)
0927 23:01:15.651037 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:15.651162 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":487,"threads_started":1}
I20260927 23:01:16.171192 19905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:16.170291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:55287)
0927 23:01:16.170868 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:16.170999 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":286,"threads_started":1}
I20260927 23:01:16.653635 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:16.652762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:36375)
0927 23:01:16.653372 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:16.653461 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":362,"threads_started":1}
I20260927 23:01:16.653733 19906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:16.652660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:44893)
0927 23:01:16.653352 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:16.653480 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":403,"threads_started":1}
I20260927 23:01:17.173622 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:17.172508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:60153)
0927 23:01:17.173213 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:17.173379 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":332,"threads_started":1}
I20260927 23:01:17.656337 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:17.654838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:50977)
0927 23:01:17.655460 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:17.656068 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":343,"threads_started":1}
I20260927 23:01:17.656607 19910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:17.655481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:57901)
0927 23:01:17.656165 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:17.656307 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":342,"threads_started":1}
I20260927 23:01:18.176018 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:18.174929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:46723)
0927 23:01:18.175642 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:18.175792 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":311,"threads_started":1}
I20260927 23:01:18.658366 19912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:18.657382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:45505)
0927 23:01:18.657984 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:18.658099 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":318,"threads_started":1}
I20260927 23:01:18.658939 19913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:18.658036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:48933)
0927 23:01:18.658601 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:18.658742 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":292,"threads_started":1}
I20260927 23:01:19.178447 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:19.177326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.130:50395)
0927 23:01:19.178009 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:19.178204 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":316,"threads_started":1}
I20260927 23:01:19.620772 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 19442
I20260927 23:01:19.654131 19915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:19.622251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.4.74.129:44205)
0927 23:01:19.623235 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:19.623256 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:19.623308 (+ 52us) server_negotiation.cc:407] Connection header received
0927 23:01:19.625676 (+ 2368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:19.625686 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:19.626154 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:19.626473 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:19.628707 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:19.630226 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:19.631824 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:19.632830 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:19.633020 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:19.645970 (+ 12950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:19.645981 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:19.646003 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:19.646062 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:19.649939 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:19.651058 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:19.651066 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:19.651074 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:19.651270 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:19.651762 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:19.651765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:19.651767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:19.652388 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:19.652637 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:19.653190 (+ 553us) server_negotiation.cc:299] Negotiation successful
0927 23:01:19.653733 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":423,"threads_started":1}
I20260927 23:01:19.655768 19600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:19.661988 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:19.659931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.131:60777)
0927 23:01:19.660909 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:19.661277 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":331,"threads_started":1}
I20260927 23:01:19.750262 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:19.746867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:43367 (local address 127.4.74.129:44029)
0927 23:01:19.749665 (+ 2798us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:19.749895 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:43367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2624,"thread_start_us":2438,"threads_started":1}
I20260927 23:01:19.750733 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 19442
I20260927 23:01:19.771436 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 19608
I20260927 23:01:19.791754 19919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:19.772828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.4.74.130:38959)
0927 23:01:19.773575 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:19.773588 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:19.773632 (+ 44us) server_negotiation.cc:407] Connection header received
0927 23:01:19.775306 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:19.775316 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:19.775835 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:19.776154 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:19.777815 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:19.779070 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:19.780555 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:19.781266 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:19.781417 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:19.784528 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:19.784536 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:19.784552 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:19.784596 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:19.788036 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:19.788893 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:19.788901 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:19.788908 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:19.789123 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:19.789514 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:19.789517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:19.789519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:19.790139 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:19.790404 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:19.790847 (+ 443us) server_negotiation.cc:299] Negotiation successful
0927 23:01:19.791353 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":411,"threads_started":1}
I20260927 23:01:19.793375 19731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:19.890628 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 19608
I20260927 23:01:19.909770 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 19739
I20260927 23:01:19.932065 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:19.911101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35026 (local address 127.4.74.131:43659)
0927 23:01:19.912012 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:19.912031 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:19.912084 (+ 53us) server_negotiation.cc:407] Connection header received
0927 23:01:19.913999 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:19.914011 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:19.914609 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:19.915056 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:19.916967 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:19.918365 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:19.919868 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:19.920686 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:19.920912 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:19.924146 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:19.924156 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:19.924178 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:19.924237 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:19.927533 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:19.928431 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:19.928439 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:19.928447 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:19.928647 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:19.929066 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:19.929079 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:19.929082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:19.929769 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:19.930333 (+ 564us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:19.930989 (+ 656us) server_negotiation.cc:299] Negotiation successful
0927 23:01:19.931673 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":489,"threads_started":1}
I20260927 23:01:19.934023 19862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:20.023681 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 19739
2026-09-27T23:01:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25898 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:01:20 dist-test-slave-5pd3 krb5kdc[19925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:01:20 dist-test-slave-5pd3 krb5kdc[19925](info): set up 2 sockets
Sep 27 23:01:20 dist-test-slave-5pd3 krb5kdc[19925](info): commencing operation
krb5kdc: starting...
W20260927 23:01:21.168300 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:01:21 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:01:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:01:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:01:21.452792 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:37973
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:37973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:01:21.862483 19938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:21.862999 19938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:21.863227 19938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:21.871891 19938 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 23:01:21.872108 19938 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 23:01:21.872218 19938 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 23:01:21.872314 19938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:01:21.872421 19938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:21.872550 19938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:21.872639 19938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:01:21.872727 19938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:01:21.885591 19938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:37973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:37973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:21.887382 19938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:21.889150 19938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:21.903820 19943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:21.903972 19944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:21.903972 19946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:21.905742 19938 server_base.cc:1034] running on GCE node
I20260927 23:01:21.906770 19938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:21.908727 19938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:21.910054 19938 hybrid_clock.cc:648] HybridClock initialized: now 1790550081909969 us; error 42 us; skew 500 ppm
I20260927 23:01:21.910493 19938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:21 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550081, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:21.914399 19938 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:01:21.916659 19938 webserver.cc:460] Webserver started at http://127.4.74.190:38965/ using document root <none> and password file <none>
I20260927 23:01:21.917702 19938 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:21.917850 19938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:21.918432 19938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:01:21.923130 19938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0"
format_stamp: "Formatted at 2026-09-27 23:01:21 on dist-test-slave-5pd3"
server_key: "8b2a4c3beb006b2c055b6e4a6c1cfa11"
server_key_iv: "2467e4934689a2d689bc866987f0f174"
server_key_version: "encryptionkey@0"
I20260927 23:01:21.924670 19938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0"
format_stamp: "Formatted at 2026-09-27 23:01:21 on dist-test-slave-5pd3"
server_key: "8b2a4c3beb006b2c055b6e4a6c1cfa11"
server_key_iv: "2467e4934689a2d689bc866987f0f174"
server_key_version: "encryptionkey@0"
I20260927 23:01:21.931352 19938 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260927 23:01:21.935787 19953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:21.938092 19938 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:01:21.938386 19938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0"
format_stamp: "Formatted at 2026-09-27 23:01:21 on dist-test-slave-5pd3"
server_key: "8b2a4c3beb006b2c055b6e4a6c1cfa11"
server_key_iv: "2467e4934689a2d689bc866987f0f174"
server_key_version: "encryptionkey@0"
I20260927 23:01:21.938671 19938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:21.960409 19938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:21.965411 19938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:21.965736 19938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:21.988430 19938 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:37973
I20260927 23:01:21.988479 20004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:37973 every 8 connection(s)
I20260927 23:01:21.990118 19938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:01:21.995314 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 19938
I20260927 23:01:21.995370 20005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:01:21.995623 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:01:21.996373 4394 external_mini_cluster.cc:1428] Setting key a1006611c12a41062f7144604636d03b
I20260927 23:01:22.011432 20005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Bootstrap starting.
Sep 27 23:01:22 dist-test-slave-5pd3 krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:01:22.015842 20005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Neither blocks nor log segments found. Creating new log.
I20260927 23:01:22.017258 20005 log.cc:824] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:22.021216 20005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: No bootstrap required, opened a new log
I20260927 23:01:22.021432 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:21.998616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59234 (local address 127.4.74.190:37973)
0927 23:01:21.999852 (+ 1236us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:21.999867 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:21.999925 (+ 58us) server_negotiation.cc:407] Connection header received
0927 23:01:22.001851 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:22.001878 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:22.002374 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:22.002852 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:22.004618 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:22.005973 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:22.007414 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:22.008312 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:22.008552 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:22.013232 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:22.013240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:22.013259 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:22.013315 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:22.016947 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:22.017783 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:22.017790 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:22.017797 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:22.017992 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:22.018392 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:22.018398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:22.018406 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:22.019018 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:22.019326 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:22.019846 (+ 520us) server_negotiation.cc:299] Negotiation successful
0927 23:01:22.020289 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":480,"threads_started":1}
I20260927 23:01:22.030692 20005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:01:22.031239 20005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:01:22.031356 20005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106ef2ae2a9a4c328a0d8ab05d269cd0, State: Initialized, Role: FOLLOWER
I20260927 23:01:22.032294 20005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:01:22.032598 20005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:01:22.032804 20005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:01:22.033007 20005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:01:22.035511 20005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:01:22.036248 20005 leader_election.cc:302] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 106ef2ae2a9a4c328a0d8ab05d269cd0; no voters:
I20260927 23:01:22.037092 20005 leader_election.cc:288] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:01:22.037292 20010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:01:22.038329 20010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 1 LEADER]: Becoming Leader. State: Replica: 106ef2ae2a9a4c328a0d8ab05d269cd0, State: Running, Role: LEADER
I20260927 23:01:22.038975 20010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:01:22.039841 20005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:01:22.042021 20012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 106ef2ae2a9a4c328a0d8ab05d269cd0. Latest consensus state: current_term: 1 leader_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } } }
I20260927 23:01:22.042405 20012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: This master's current role is: LEADER
I20260927 23:01:22.043530 20016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:01:22.043324 20011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } } }
I20260927 23:01:22.043748 20011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: This master's current role is: LEADER
I20260927 23:01:22.049819 20016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:01:22.062484 20016 catalog_manager.cc:1347] Generated new cluster ID: 5e2eae7bdbe048ed992a2bc7427a3d2d
I20260927 23:01:22.062641 20016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:01:22.087038 20016 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:01:22.088992 20016 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:01:22.098429 20016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Generated new TSK 0
I20260927 23:01:22.099772 20016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 23:01:22.168129 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:37973
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:01:22.592813 20031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:22.593194 20031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:22.593370 20031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:22.601835 20031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:22.601994 20031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:22.602212 20031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:01:22.613816 20031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:22.615721 20031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:22.617504 20031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:22.634460 20037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:22.634562 20039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:22.634783 20036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:22.635782 20031 server_base.cc:1034] running on GCE node
I20260927 23:01:22.636824 20031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:22.638253 20031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:22.639516 20031 hybrid_clock.cc:648] HybridClock initialized: now 1790550082639451 us; error 71 us; skew 500 ppm
I20260927 23:01:22.639993 20031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:22 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550082, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:22.644055 20031 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:01:22.646214 20031 webserver.cc:460] Webserver started at http://127.4.74.129:37369/ using document root <none> and password file <none>
I20260927 23:01:22.647154 20031 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:22.647325 20031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:22.647771 20031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:01:22.651315 20031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "17e750d3cd76428b90c8b2f01ac81ee1"
format_stamp: "Formatted at 2026-09-27 23:01:22 on dist-test-slave-5pd3"
server_key: "80be0a2e76b09548ff891c78f5861fc3"
server_key_iv: "7fabb5123cfaf4404508f0511f1f322e"
server_key_version: "encryptionkey@0"
I20260927 23:01:22.652304 20031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "17e750d3cd76428b90c8b2f01ac81ee1"
format_stamp: "Formatted at 2026-09-27 23:01:22 on dist-test-slave-5pd3"
server_key: "80be0a2e76b09548ff891c78f5861fc3"
server_key_iv: "7fabb5123cfaf4404508f0511f1f322e"
server_key_version: "encryptionkey@0"
I20260927 23:01:22.658936 20031 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260927 23:01:22.663511 20046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:22.665380 20031 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:01:22.665623 20031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "17e750d3cd76428b90c8b2f01ac81ee1"
format_stamp: "Formatted at 2026-09-27 23:01:22 on dist-test-slave-5pd3"
server_key: "80be0a2e76b09548ff891c78f5861fc3"
server_key_iv: "7fabb5123cfaf4404508f0511f1f322e"
server_key_version: "encryptionkey@0"
I20260927 23:01:22.665858 20031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:22.680862 20031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:22.686055 20031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:22.686362 20031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:22.687515 20031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:22.689220 20031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:22.689302 20031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:22.689399 20031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:22.689448 20031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:22.730633 20031 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:44679
I20260927 23:01:22.730748 20158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:44679 every 8 connection(s)
I20260927 23:01:22.732247 20031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:01:22.741225 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 20031
I20260927 23:01:22.741487 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 23:01:22.742238 4394 external_mini_cluster.cc:1428] Setting key aa9420045c9abf62d5a33652dfac35e9
Sep 27 23:01:22 dist-test-slave-5pd3 krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550082, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:01:22.758490 20161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:22.735438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:36323 (local address 127.4.74.190:37973)
0927 23:01:22.736075 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:22.736085 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:22.737333 (+ 1248us) server_negotiation.cc:407] Connection header received
0927 23:01:22.739412 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:22.739417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:22.739510 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:22.739703 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:22.741929 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:22.742951 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:22.745254 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:22.745713 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:22.745850 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:22.751429 (+ 5579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:22.751436 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:22.751440 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:22.751487 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:22.754577 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:22.755648 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:22.755653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:22.755656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:22.755803 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:22.756310 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:22.756318 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:22.756322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:22.756644 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:22.756820 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:22.757764 (+ 944us) server_negotiation.cc:299] Negotiation successful
0927 23:01:22.758075 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":308,"threads_started":1}
I20260927 23:01:22.759974 20162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:22.736173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:36323)
0927 23:01:22.737118 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:22.737166 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:22.739108 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:22.740050 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:22.740060 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:22.740711 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:01:22.741654 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:22.741684 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:22.743186 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:22.743192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:22.745062 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:22.745080 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:22.745999 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:22.746008 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:22.746359 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:22.747529 (+ 1170us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:01:22.747602 (+ 73us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:01:22.751052 (+ 3450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:01:22.754807 (+ 3755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:22.754821 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:22.754851 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:22.755456 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:22.755932 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:22.755937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:22.755942 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:22.756173 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:22.756790 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:01:22.756800 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:01:22.757306 (+ 506us) client_negotiation.cc:769] Sending connection context
0927 23:01:22.757633 (+ 327us) client_negotiation.cc:240] Negotiation successful
0927 23:01:22.758356 (+ 723us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":337,"threads_started":1}
I20260927 23:01:22.762260 20159 heartbeater.cc:347] Connected to a master server at 127.4.74.190:37973
I20260927 23:01:22.762706 20159 heartbeater.cc:464] Registering TS with master...
I20260927 23:01:22.763798 20159 heartbeater.cc:511] Master 127.4.74.190:37973 requested a full tablet report, sending...
I20260927 23:01:22.766949 19970 ts_manager.cc:194] Registered new tserver with Master: 17e750d3cd76428b90c8b2f01ac81ee1 (127.4.74.129:44679)
I20260927 23:01:22.768967 19970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:36323
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 23:01:22.802877 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:37973
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:01:23.164086 20165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:23.164448 20165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:23.164578 20165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:23.173115 20165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:23.173236 20165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:23.173414 20165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:01:23.183494 20165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:23.184981 20165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:23.186322 20165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:23.200896 20170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:23.201023 20171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:23.201064 20173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:23.203125 20165 server_base.cc:1034] running on GCE node
I20260927 23:01:23.203994 20165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:23.205191 20165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:23.206597 20165 hybrid_clock.cc:648] HybridClock initialized: now 1790550083206523 us; error 66 us; skew 500 ppm
I20260927 23:01:23.207028 20165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:23 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:23.211345 20165 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:01:23.213359 20165 webserver.cc:460] Webserver started at http://127.4.74.130:45233/ using document root <none> and password file <none>
I20260927 23:01:23.214190 20165 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:23.214306 20165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:23.214694 20165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:01:23.217918 20165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "0afe24ae78594a2c8abf62313638fc1e"
format_stamp: "Formatted at 2026-09-27 23:01:23 on dist-test-slave-5pd3"
server_key: "6c98633743b5ccd3d22bdae25001af31"
server_key_iv: "7f49210dc934f3be90b62245781ad7c5"
server_key_version: "encryptionkey@0"
I20260927 23:01:23.218751 20165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "0afe24ae78594a2c8abf62313638fc1e"
format_stamp: "Formatted at 2026-09-27 23:01:23 on dist-test-slave-5pd3"
server_key: "6c98633743b5ccd3d22bdae25001af31"
server_key_iv: "7f49210dc934f3be90b62245781ad7c5"
server_key_version: "encryptionkey@0"
I20260927 23:01:23.225188 20165 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260927 23:01:23.229839 20180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:23.231793 20165 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260927 23:01:23.232035 20165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "0afe24ae78594a2c8abf62313638fc1e"
format_stamp: "Formatted at 2026-09-27 23:01:23 on dist-test-slave-5pd3"
server_key: "6c98633743b5ccd3d22bdae25001af31"
server_key_iv: "7f49210dc934f3be90b62245781ad7c5"
server_key_version: "encryptionkey@0"
I20260927 23:01:23.232286 20165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:23.252955 20165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:23.257869 20165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:23.258225 20165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:23.259532 20165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:23.261255 20165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:23.261360 20165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:23.261515 20165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:23.261587 20165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:23.300247 20165 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:43463
I20260927 23:01:23.300374 20292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:43463 every 8 connection(s)
I20260927 23:01:23.301782 20165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:01:23.303429 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 20165
I20260927 23:01:23.303655 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 23:01:23.304522 4394 external_mini_cluster.cc:1428] Setting key 46b2491d699fe6f9f801f0c87a2b851b
Sep 27 23:01:23 dist-test-slave-5pd3 krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:01:23.334321 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.305706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:34293 (local address 127.4.74.190:37973)
0927 23:01:23.312042 (+ 6336us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:23.312047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:23.312070 (+ 23us) server_negotiation.cc:407] Connection header received
0927 23:01:23.312137 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:23.312141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:23.312217 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:23.312343 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:23.315268 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.316170 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.319281 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.319836 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.319961 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.325013 (+ 5052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:23.325022 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:23.325030 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:23.325091 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:23.329249 (+ 4158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:23.330588 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:23.330597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:23.330603 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:23.330800 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:23.331597 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:23.331605 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:23.331612 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:23.332120 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:23.332289 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:23.333427 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0927 23:01:23.333745 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6192,"thread_start_us":6062,"threads_started":1}
I20260927 23:01:23.335275 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.306419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:34293)
0927 23:01:23.307802 (+ 1383us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:23.308009 (+ 207us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:23.311532 (+ 3523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:23.312767 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:23.312786 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:23.313846 (+ 1060us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:01:23.314987 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:23.315021 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.316389 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.316400 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:23.319065 (+ 2665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:23.319091 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.320157 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.320168 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:23.320570 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.321506 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:01:23.321536 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:01:23.324636 (+ 3100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:01:23.329535 (+ 4899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:23.329553 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:23.329581 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:23.330330 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:23.331103 (+ 773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:23.331108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:23.331110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:23.331411 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:23.332299 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:01:23.332307 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:01:23.332846 (+ 539us) client_negotiation.cc:769] Sending connection context
0927 23:01:23.333265 (+ 419us) client_negotiation.cc:240] Negotiation successful
0927 23:01:23.333843 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"thread_start_us":486,"threads_started":1}
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 23:01:23.337410 20293 heartbeater.cc:347] Connected to a master server at 127.4.74.190:37973
I20260927 23:01:23.337841 20293 heartbeater.cc:464] Registering TS with master...
I20260927 23:01:23.338872 20293 heartbeater.cc:511] Master 127.4.74.190:37973 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
I20260927 23:01:23.341032 19970 ts_manager.cc:194] Registered new tserver with Master: 0afe24ae78594a2c8abf62313638fc1e (127.4.74.130:43463)
I20260927 23:01:23.342054 19970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:34293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 23:01:23.372179 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:37973
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:01:23.715066 20299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:23.715428 20299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:23.715665 20299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:23.723721 20299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:23.723897 20299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:23.724092 20299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:01:23.737028 20299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:23.738485 20299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:23.740144 20299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:23.756664 20304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:23.756740 20305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:23.757619 20307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:23.758872 20299 server_base.cc:1034] running on GCE node
I20260927 23:01:23.759728 20299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:23.761000 20299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:23.762259 20299 hybrid_clock.cc:648] HybridClock initialized: now 1790550083762202 us; error 55 us; skew 500 ppm
I20260927 23:01:23.762629 20299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:23 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:23.766037 20299 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:01:23.767849 20299 webserver.cc:460] Webserver started at http://127.4.74.131:42053/ using document root <none> and password file <none>
I20260927 23:01:23.768575 20299 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:23.768682 20299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:23.768980 20299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:01:23.772284 20299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "058709d6b21d4655b8d9708507719c26"
format_stamp: "Formatted at 2026-09-27 23:01:23 on dist-test-slave-5pd3"
server_key: "36b14638323b01919ef39505faed0496"
server_key_iv: "eeb1af63a65fc3abe5f0ff43e878e696"
server_key_version: "encryptionkey@0"
I20260927 23:01:23.773025 20299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "058709d6b21d4655b8d9708507719c26"
format_stamp: "Formatted at 2026-09-27 23:01:23 on dist-test-slave-5pd3"
server_key: "36b14638323b01919ef39505faed0496"
server_key_iv: "eeb1af63a65fc3abe5f0ff43e878e696"
server_key_version: "encryptionkey@0"
I20260927 23:01:23.773593 20159 heartbeater.cc:503] Master 127.4.74.190:37973 was elected leader, sending a full tablet report...
I20260927 23:01:23.779645 20299 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260927 23:01:23.784134 20314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:23.786311 20299 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:01:23.786667 20299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "058709d6b21d4655b8d9708507719c26"
format_stamp: "Formatted at 2026-09-27 23:01:23 on dist-test-slave-5pd3"
server_key: "36b14638323b01919ef39505faed0496"
server_key_iv: "eeb1af63a65fc3abe5f0ff43e878e696"
server_key_version: "encryptionkey@0"
I20260927 23:01:23.786912 20299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:23.825579 20299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:23.830698 20299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:23.831035 20299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:23.832239 20299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:23.833755 20299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:23.833844 20299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:23.834043 20299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:23.834101 20299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:23.874893 20299 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:36535
I20260927 23:01:23.875042 20426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:36535 every 8 connection(s)
I20260927 23:01:23.876330 20299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:01:23.882294 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 20299
I20260927 23:01:23.882552 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 23:01:23.883251 4394 external_mini_cluster.cc:1428] Setting key 1c9b6c1218112bbbb4d9bf2fd0c72ebc
Sep 27 23:01:23 dist-test-slave-5pd3 krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550083, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:01:23.903915 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.879717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:53729 (local address 127.4.74.190:37973)
0927 23:01:23.880321 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:23.880326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:23.881798 (+ 1472us) server_negotiation.cc:407] Connection header received
0927 23:01:23.884974 (+ 3176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:23.884980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:23.885165 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:23.885311 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:23.887460 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.888547 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.890529 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.891036 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.891195 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.895978 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:23.895985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:23.895989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:23.896038 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:23.899804 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:23.901091 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:23.901095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:23.901098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:23.901265 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:23.901736 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:23.901739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:23.901741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:23.902042 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:23.902234 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:23.903293 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0927 23:01:23.903533 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":282,"threads_started":1}
I20260927 23:01:23.905290 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.880305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:53729)
0927 23:01:23.881470 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:23.881532 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:23.884560 (+ 3028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:23.885659 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:23.885671 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:23.886377 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:01:23.887211 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:23.887237 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.888765 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.888769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:23.890310 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:23.890326 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.891349 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.891358 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:23.891716 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.892617 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:01:23.892650 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:01:23.895499 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:01:23.900027 (+ 4528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:23.900041 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:23.900067 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:23.900944 (+ 877us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:23.901384 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:23.901388 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:23.901391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:23.901626 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:23.902242 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:01:23.902250 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:01:23.902810 (+ 560us) client_negotiation.cc:769] Sending connection context
0927 23:01:23.903134 (+ 324us) client_negotiation.cc:240] Negotiation successful
0927 23:01:23.903820 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":393,"threads_started":1}
I20260927 23:01:23.907274 20427 heartbeater.cc:347] Connected to a master server at 127.4.74.190:37973
I20260927 23:01:23.907797 20427 heartbeater.cc:464] Registering TS with master...
I20260927 23:01:23.908831 20427 heartbeater.cc:511] Master 127.4.74.190:37973 requested a full tablet report, sending...
I20260927 23:01:23.911110 19970 ts_manager.cc:194] Registered new tserver with Master: 058709d6b21d4655b8d9708507719c26 (127.4.74.131:36535)
I20260927 23:01:23.912416 19970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:53729
I20260927 23:01:23.913980 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:01:23.934746 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.918969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59242 (local address 127.4.74.190:37973)
0927 23:01:23.919211 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:23.919216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:23.919532 (+ 316us) server_negotiation.cc:407] Connection header received
0927 23:01:23.919782 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:23.919786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:23.919878 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:23.920005 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:23.921635 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.922545 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.924275 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.924756 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.924921 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.927880 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:23.927887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:23.927891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:23.927947 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:23.931455 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:23.932464 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:23.932468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:23.932471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:23.932636 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:23.933139 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:23.933142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:23.933144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:23.933567 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:23.933788 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:23.934188 (+ 400us) server_negotiation.cc:299] Negotiation successful
0927 23:01:23.934453 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 23:01:23.940186 19970 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:01:23.948828 19970 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:01:23.980790 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.969344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.4.74.129:44679)
0927 23:01:23.970670 (+ 1326us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:23.970725 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:23.971314 (+ 589us) server_negotiation.cc:407] Connection header received
0927 23:01:23.971464 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:23.971473 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:23.971795 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:23.972047 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:23.972889 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.974694 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.977774 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.979193 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.979361 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.979687 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:23.980143 (+ 456us) server_negotiation.cc:299] Negotiation successful
0927 23:01:23.980351 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1118,"thread_start_us":945,"threads_started":1}
I20260927 23:01:23.980880 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.963960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:36535 (local address 127.0.0.1:46134)
0927 23:01:23.969293 (+ 5333us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:23.969363 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:23.969631 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:23.971063 (+ 1432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:23.971072 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:23.971116 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:23.971688 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:23.971705 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.974863 (+ 3158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.974871 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:23.977759 (+ 2888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:23.977780 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.979876 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.979885 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:23.980094 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.980128 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 23:01:23.980238 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 23:01:23.980429 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4979,"mutex_wait_us":92,"thread_start_us":4413,"threads_started":1}
I20260927 23:01:23.981418 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.964317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46134 (local address 127.4.74.131:36535)
0927 23:01:23.970181 (+ 5864us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:23.970194 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:23.970243 (+ 49us) server_negotiation.cc:407] Connection header received
0927 23:01:23.970394 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:23.970407 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:23.970798 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:23.971063 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:23.971928 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.974686 (+ 2758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.978041 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.979652 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.979810 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.980078 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:23.980345 (+ 267us) server_negotiation.cc:299] Negotiation successful
0927 23:01:23.980542 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5572,"thread_start_us":5362,"threads_started":1}
I20260927 23:01:23.981547 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.964897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.4.74.130:43463)
0927 23:01:23.966552 (+ 1655us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:23.966565 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:23.971334 (+ 4769us) server_negotiation.cc:407] Connection header received
0927 23:01:23.971472 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:23.971479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:23.971863 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:23.972047 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:23.973231 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.974466 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.978004 (+ 3538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.979644 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.979809 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.980038 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:23.980992 (+ 954us) server_negotiation.cc:299] Negotiation successful
0927 23:01:23.981184 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1136,"thread_start_us":857,"threads_started":1}
I20260927 23:01:23.981626 20445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.968893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.0.0.1:44268)
0927 23:01:23.970947 (+ 2054us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:23.970971 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:23.971162 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:23.972040 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:23.972044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:23.972069 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:23.972557 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:23.972574 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.975060 (+ 2486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.975066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:23.977515 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:23.977535 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.979430 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.979436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:23.979700 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.979768 (+ 68us) client_negotiation.cc:769] Sending connection context
0927 23:01:23.979935 (+ 167us) client_negotiation.cc:240] Negotiation successful
0927 23:01:23.981180 (+ 1245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1909,"mutex_wait_us":2056,"thread_start_us":691,"threads_started":1}
I20260927 23:01:23.982160 20440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:23.964043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:43463 (local address 127.0.0.1:42964)
0927 23:01:23.970947 (+ 6904us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:23.970971 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:23.971173 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:23.972330 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:23.972335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:23.972364 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:23.973012 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:23.973025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.974622 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.974630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:23.977759 (+ 3129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:23.977780 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:23.980263 (+ 2483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:23.980273 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:23.980481 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:23.980527 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 23:01:23.980636 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 23:01:23.980829 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5869,"mutex_wait_us":167,"thread_start_us":2790,"threads_started":1}
I20260927 23:01:23.986677 20094 tablet_service.cc:1467] Processing CreateTablet for tablet 1acfaa7d4d3b485e98539b8cb044b4f4 (DEFAULT_TABLE table=test-table [id=d083e6c08773499594d4da1dbb7c6afd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:01:23.986960 20362 tablet_service.cc:1467] Processing CreateTablet for tablet 1acfaa7d4d3b485e98539b8cb044b4f4 (DEFAULT_TABLE table=test-table [id=d083e6c08773499594d4da1dbb7c6afd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:01:23.986960 20228 tablet_service.cc:1467] Processing CreateTablet for tablet 1acfaa7d4d3b485e98539b8cb044b4f4 (DEFAULT_TABLE table=test-table [id=d083e6c08773499594d4da1dbb7c6afd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:01:23.988503 20094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1acfaa7d4d3b485e98539b8cb044b4f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:01:23.988652 20228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1acfaa7d4d3b485e98539b8cb044b4f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:01:23.988647 20362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1acfaa7d4d3b485e98539b8cb044b4f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:01:24.015681 20447 tablet_bootstrap.cc:492] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: Bootstrap starting.
I20260927 23:01:24.018831 20447 tablet_bootstrap.cc:654] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: Neither blocks nor log segments found. Creating new log.
I20260927 23:01:24.019722 20446 tablet_bootstrap.cc:492] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: Bootstrap starting.
I20260927 23:01:24.020115 20447 log.cc:824] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:24.024628 20448 tablet_bootstrap.cc:492] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: Bootstrap starting.
I20260927 23:01:24.026649 20446 tablet_bootstrap.cc:654] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: Neither blocks nor log segments found. Creating new log.
I20260927 23:01:24.028266 20446 log.cc:824] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:24.028578 20448 tablet_bootstrap.cc:654] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: Neither blocks nor log segments found. Creating new log.
I20260927 23:01:24.029419 20447 tablet_bootstrap.cc:492] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: No bootstrap required, opened a new log
I20260927 23:01:24.029747 20447 ts_tablet_manager.cc:1397] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260927 23:01:24.029930 20448 log.cc:824] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:24.032806 20446 tablet_bootstrap.cc:492] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: No bootstrap required, opened a new log
I20260927 23:01:24.033160 20446 ts_tablet_manager.cc:1397] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.000s
I20260927 23:01:24.033447 20448 tablet_bootstrap.cc:492] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: No bootstrap required, opened a new log
I20260927 23:01:24.033736 20448 ts_tablet_manager.cc:1397] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260927 23:01:24.041170 20447 raft_consensus.cc:348] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } } peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } }
I20260927 23:01:24.041679 20447 raft_consensus.cc:374] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:01:24.041965 20447 raft_consensus.cc:729] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058709d6b21d4655b8d9708507719c26, State: Initialized, Role: FOLLOWER
I20260927 23:01:24.042539 20446 raft_consensus.cc:348] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } } peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } }
I20260927 23:01:24.042866 20447 consensus_queue.cc:260] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } } peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } }
I20260927 23:01:24.043078 20446 raft_consensus.cc:374] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:01:24.043213 20446 raft_consensus.cc:729] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0afe24ae78594a2c8abf62313638fc1e, State: Initialized, Role: FOLLOWER
I20260927 23:01:24.043977 20446 consensus_queue.cc:260] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } } peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } }
I20260927 23:01:24.044730 20448 raft_consensus.cc:348] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } } peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } }
I20260927 23:01:24.045282 20293 heartbeater.cc:503] Master 127.4.74.190:37973 was elected leader, sending a full tablet report...
I20260927 23:01:24.045313 20448 raft_consensus.cc:374] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:01:24.045431 20448 raft_consensus.cc:729] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e750d3cd76428b90c8b2f01ac81ee1, State: Initialized, Role: FOLLOWER
I20260927 23:01:24.046099 20446 ts_tablet_manager.cc:1428] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260927 23:01:24.046360 20448 consensus_queue.cc:260] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } } peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } }
I20260927 23:01:24.047487 20427 heartbeater.cc:503] Master 127.4.74.190:37973 was elected leader, sending a full tablet report...
I20260927 23:01:24.048702 20447 ts_tablet_manager.cc:1428] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: Time spent starting tablet: real 0.019s user 0.014s sys 0.004s
I20260927 23:01:24.050398 20448 ts_tablet_manager.cc:1428] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: Time spent starting tablet: real 0.017s user 0.001s sys 0.016s
W20260927 23:01:24.054263 20294 tablet.cc:2369] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 23:01:24.128252 20428 tablet.cc:2369] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:01:24.140657 20453 raft_consensus.cc:482] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:01:24.141147 20453 raft_consensus.cc:504] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } } peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } }
I20260927 23:01:24.143452 20453 leader_election.cc:288] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 058709d6b21d4655b8d9708507719c26 (127.4.74.131:36535), 17e750d3cd76428b90c8b2f01ac81ee1 (127.4.74.129:44679)
I20260927 23:01:24.151925 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:24.144065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:44467 (local address 127.4.74.131:36535)
0927 23:01:24.144345 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:24.144354 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:24.144958 (+ 604us) server_negotiation.cc:407] Connection header received
0927 23:01:24.145255 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:24.145261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:24.145329 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:24.145450 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:24.146294 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.147506 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.150018 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.150869 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.151077 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:24.151161 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:24.151472 (+ 311us) server_negotiation.cc:299] Negotiation successful
0927 23:01:24.151651 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 23:01:24.152375 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:24.143884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:36535 (local address 127.4.74.130:44467)
0927 23:01:24.144756 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:24.144803 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:24.145094 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:24.145635 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:24.145640 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:24.145685 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:24.146045 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:24.146078 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.147732 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.147739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:24.149812 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:24.149828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.151118 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.151124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:24.151299 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:24.151326 (+ 27us) client_negotiation.cc:769] Sending connection context
0927 23:01:24.151444 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 23:01:24.151612 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":75,"thread_start_us":515,"threads_started":1}
I20260927 23:01:24.152878 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:24.145286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:60727 (local address 127.4.74.129:44679)
0927 23:01:24.145555 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:24.145562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:24.146109 (+ 547us) server_negotiation.cc:407] Connection header received
0927 23:01:24.146379 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:24.146390 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:24.146480 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:24.146571 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:24.147378 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.148601 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.150824 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.151817 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.152030 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:24.152123 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:24.152438 (+ 315us) server_negotiation.cc:299] Negotiation successful
0927 23:01:24.152592 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260927 23:01:24.153020 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:24.144847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.4.74.130:60727)
0927 23:01:24.145967 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:24.145989 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:24.146144 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:24.146632 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:24.146638 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:24.146678 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:24.147166 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:24.147179 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.148771 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.148775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:24.150580 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:24.150595 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.152076 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.152082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:24.152257 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:24.152282 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 23:01:24.152407 (+ 125us) client_negotiation.cc:240] Negotiation successful
0927 23:01:24.152562 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"mutex_wait_us":7,"thread_start_us":740,"threads_started":1}
I20260927 23:01:24.154145 20382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1acfaa7d4d3b485e98539b8cb044b4f4" candidate_uuid: "0afe24ae78594a2c8abf62313638fc1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058709d6b21d4655b8d9708507719c26" is_pre_election: true
I20260927 23:01:24.154223 20114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1acfaa7d4d3b485e98539b8cb044b4f4" candidate_uuid: "0afe24ae78594a2c8abf62313638fc1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" is_pre_election: true
I20260927 23:01:24.154706 20382 raft_consensus.cc:2454] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0afe24ae78594a2c8abf62313638fc1e in term 0.
I20260927 23:01:24.154867 20114 raft_consensus.cc:2454] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0afe24ae78594a2c8abf62313638fc1e in term 0.
I20260927 23:01:24.155478 20181 leader_election.cc:302] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058709d6b21d4655b8d9708507719c26, 0afe24ae78594a2c8abf62313638fc1e; no voters:
I20260927 23:01:24.156045 20453 raft_consensus.cc:2781] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:01:24.156234 20453 raft_consensus.cc:482] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:01:24.156379 20453 raft_consensus.cc:3037] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:01:24.158874 20453 raft_consensus.cc:504] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } } peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } }
I20260927 23:01:24.159787 20453 leader_election.cc:288] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [CANDIDATE]: Term 1 election: Requested vote from peers 058709d6b21d4655b8d9708507719c26 (127.4.74.131:36535), 17e750d3cd76428b90c8b2f01ac81ee1 (127.4.74.129:44679)
I20260927 23:01:24.160333 20382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1acfaa7d4d3b485e98539b8cb044b4f4" candidate_uuid: "0afe24ae78594a2c8abf62313638fc1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058709d6b21d4655b8d9708507719c26"
I20260927 23:01:24.160540 20114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1acfaa7d4d3b485e98539b8cb044b4f4" candidate_uuid: "0afe24ae78594a2c8abf62313638fc1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e750d3cd76428b90c8b2f01ac81ee1"
I20260927 23:01:24.160646 20382 raft_consensus.cc:3037] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:01:24.160835 20114 raft_consensus.cc:3037] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:01:24.163012 20382 raft_consensus.cc:2454] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0afe24ae78594a2c8abf62313638fc1e in term 1.
I20260927 23:01:24.163407 20114 raft_consensus.cc:2454] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0afe24ae78594a2c8abf62313638fc1e in term 1.
I20260927 23:01:24.163517 20181 leader_election.cc:302] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058709d6b21d4655b8d9708507719c26, 0afe24ae78594a2c8abf62313638fc1e; no voters:
I20260927 23:01:24.164011 20453 raft_consensus.cc:2781] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:01:24.164625 20453 raft_consensus.cc:686] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 1 LEADER]: Becoming Leader. State: Replica: 0afe24ae78594a2c8abf62313638fc1e, State: Running, Role: LEADER
I20260927 23:01:24.165409 20453 consensus_queue.cc:237] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } } peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } }
I20260927 23:01:24.171591 19970 catalog_manager.cc:5495] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0afe24ae78594a2c8abf62313638fc1e (127.4.74.130). New cstate: current_term: 1 leader_uuid: "0afe24ae78594a2c8abf62313638fc1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } health_report { overall_health: UNKNOWN } } }
I20260927 23:01:24.228605 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:24.221722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.4.74.130:43463)
0927 23:01:24.222013 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:24.222021 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:24.222045 (+ 24us) server_negotiation.cc:407] Connection header received
0927 23:01:24.222333 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:24.222337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:24.222444 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:24.222626 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:01:24.223057 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.224089 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.225796 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.226233 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.226384 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:24.227020 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:01:24.227856 (+ 836us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:01:24.228032 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:24.228131 (+ 99us) server_negotiation.cc:299] Negotiation successful
0927 23:01:24.228252 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
W20260927 23:01:24.235517 20160 tablet.cc:2369] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:01:24.239827 20382 raft_consensus.cc:1261] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26 [term 1 FOLLOWER]: Refusing update from remote peer 0afe24ae78594a2c8abf62313638fc1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:01:24.239827 20114 raft_consensus.cc:1261] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 1 FOLLOWER]: Refusing update from remote peer 0afe24ae78594a2c8abf62313638fc1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:01:24.241258 20457 consensus_queue.cc:1035] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:01:24.241793 20453 consensus_queue.cc:1035] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:01:24.256172 20460 mvcc.cc:205] Tried to move back new op lower bound from 7334093145030115328 to 7334093144746754048. Current Snapshot: MvccSnapshot[applied={T|T < 7334093145030115328}]
I20260927 23:01:24.276969 19967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:01:24.277338 19967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:01:24.281703 19967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Sending DeleteTablet for 3 replicas of tablet 1acfaa7d4d3b485e98539b8cb044b4f4
I20260927 23:01:24.283397 20362 tablet_service.cc:1514] Processing DeleteTablet for tablet 1acfaa7d4d3b485e98539b8cb044b4f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:01:24 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:46134
I20260927 23:01:24.283404 20228 tablet_service.cc:1514] Processing DeleteTablet for tablet 1acfaa7d4d3b485e98539b8cb044b4f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:01:24 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:42964
I20260927 23:01:24.283797 20094 tablet_service.cc:1514] Processing DeleteTablet for tablet 1acfaa7d4d3b485e98539b8cb044b4f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:01:24 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:44268
I20260927 23:01:24.287156 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 20031
I20260927 23:01:24.295500 20472 tablet_replica.cc:330] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: stopping tablet replica
I20260927 23:01:24.296345 20472 raft_consensus.cc:2229] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:01:24.297042 20472 raft_consensus.cc:2258] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:01:24.297999 20470 tablet_replica.cc:330] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: stopping tablet replica
I20260927 23:01:24.298506 20470 raft_consensus.cc:2229] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:01:24.299105 20470 raft_consensus.cc:2258] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:01:24.300982 20471 tablet_replica.cc:330] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: stopping tablet replica
I20260927 23:01:24.302323 20472 ts_tablet_manager.cc:1905] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:01:24.302932 20470 ts_tablet_manager.cc:1905] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:01:24.301514 20471 raft_consensus.cc:2229] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 1 LEADER]: Raft consensus shutting down.
Sep 27 23:01:24 dist-test-slave-5pd3 krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 23:01:24.310564 20472 ts_tablet_manager.cc:1918] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:01:24.310802 20472 log.cc:1196] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/1acfaa7d4d3b485e98539b8cb044b4f4
I20260927 23:01:24.311218 20470 ts_tablet_manager.cc:1918] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:01:24.311409 20470 log.cc:1196] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/1acfaa7d4d3b485e98539b8cb044b4f4
I20260927 23:01:24.311422 20472 ts_tablet_manager.cc:1939] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 17e750d3cd76428b90c8b2f01ac81ee1: Deleting consensus metadata
I20260927 23:01:24.311970 20470 ts_tablet_manager.cc:1939] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 058709d6b21d4655b8d9708507719c26: Deleting consensus metadata
I20260927 23:01:24.314039 19954 catalog_manager.cc:4841] TS 058709d6b21d4655b8d9708507719c26 (127.4.74.131:36535): tablet 1acfaa7d4d3b485e98539b8cb044b4f4 (table test-table [id=d083e6c08773499594d4da1dbb7c6afd]) successfully deleted
I20260927 23:01:24.314407 19954 catalog_manager.cc:4841] TS 17e750d3cd76428b90c8b2f01ac81ee1 (127.4.74.129:44679): tablet 1acfaa7d4d3b485e98539b8cb044b4f4 (table test-table [id=d083e6c08773499594d4da1dbb7c6afd]) successfully deleted
I20260927 23:01:24.314531 20471 raft_consensus.cc:2258] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:01:24.317999 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:24.288956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44276 (local address 127.4.74.129:44679)
0927 23:01:24.289216 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:24.289225 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:24.298596 (+ 9371us) server_negotiation.cc:407] Connection header received
0927 23:01:24.298919 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:24.298926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:24.299035 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:24.299146 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:24.301548 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.302376 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.304142 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.304612 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.304770 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:24.310357 (+ 5587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:24.310369 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:24.310383 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:24.310424 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:24.314532 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:24.315546 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:24.315602 (+ 56us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:24.315606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:24.315783 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:24.316166 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:24.316171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:24.316175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:24.316669 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:24.316985 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:24.317356 (+ 371us) server_negotiation.cc:299] Negotiation successful
0927 23:01:24.317592 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260927 23:01:24.318763 20154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:24.320269 20471 ts_tablet_manager.cc:1905] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:01:24.329295 20471 ts_tablet_manager.cc:1918] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:01:24.329557 20471 log.cc:1196] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/1acfaa7d4d3b485e98539b8cb044b4f4
I20260927 23:01:24.330175 20471 ts_tablet_manager.cc:1939] T 1acfaa7d4d3b485e98539b8cb044b4f4 P 0afe24ae78594a2c8abf62313638fc1e: Deleting consensus metadata
I20260927 23:01:24.332650 19956 catalog_manager.cc:4841] TS 0afe24ae78594a2c8abf62313638fc1e (127.4.74.130:43463): tablet 1acfaa7d4d3b485e98539b8cb044b4f4 (table test-table [id=d083e6c08773499594d4da1dbb7c6afd]) successfully deleted
I20260927 23:01:24.424449 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 20031
I20260927 23:01:24.453014 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 20165
Sep 27 23:01:24 dist-test-slave-5pd3 krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 23:01:24.474946 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:24.454102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42978 (local address 127.4.74.130:43463)
0927 23:01:24.454336 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:24.454341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:24.454361 (+ 20us) server_negotiation.cc:407] Connection header received
0927 23:01:24.454659 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:24.454664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:24.454739 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:24.454893 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:24.456959 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.457880 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.460352 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.460847 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.461016 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:24.467022 (+ 6006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:24.467039 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:24.467056 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:24.467101 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:24.470828 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:24.471710 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:24.471717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:24.471720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:24.471945 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:24.472380 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:24.472384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:24.472389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:24.472781 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:24.473127 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:24.474240 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0927 23:01:24.474634 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 23:01:24.475891 20288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:24.608868 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 20165
I20260927 23:01:24.628916 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 20299
Sep 27 23:01:24 dist-test-slave-5pd3 krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 23:01:24.655946 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:24.629827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46140 (local address 127.4.74.131:36535)
0927 23:01:24.630046 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:24.630051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:24.630138 (+ 87us) server_negotiation.cc:407] Connection header received
0927 23:01:24.630336 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:24.630340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:24.630432 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:24.630553 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:24.632186 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.632984 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.635040 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:24.635483 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:24.635860 (+ 377us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:24.641604 (+ 5744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:24.641629 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:24.641650 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:24.641713 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:24.652710 (+ 10997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:24.653775 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:24.653783 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:24.653786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:24.653970 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:24.654345 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:24.654348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:24.654350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:24.654768 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:24.655023 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:24.655332 (+ 309us) server_negotiation.cc:299] Negotiation successful
0927 23:01:24.655600 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 23:01:24.656849 20422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:24.753954 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 20299
I20260927 23:01:24.773054 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 19938
I20260927 23:01:24.774509 20000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:24.904222 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 19938
I20260927 23:01:44.931347 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:37973
--webserver_interface=127.4.74.190
--webserver_port=38965
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:37973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:01:45.277561 20478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:45.277920 20478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:45.278075 20478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:45.285983 20478 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 23:01:45.286108 20478 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 23:01:45.286150 20478 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 23:01:45.286244 20478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:01:45.286345 20478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:45.286505 20478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:45.286564 20478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:01:45.286646 20478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:01:45.297145 20478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:37973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:37973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=38965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:45.298594 20478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:45.299999 20478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:45.327733 20484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:45.327817 20487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:45.327798 20485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:45.328982 20478 server_base.cc:1034] running on GCE node
I20260927 23:01:45.330058 20478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:45.332280 20478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:45.333590 20478 hybrid_clock.cc:648] HybridClock initialized: now 1790550105333490 us; error 73 us; skew 500 ppm
I20260927 23:01:45.334076 20478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:45 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550105, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:45.338361 20478 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:01:45.340489 20478 webserver.cc:460] Webserver started at http://127.4.74.190:38965/ using document root <none> and password file <none>
I20260927 23:01:45.341356 20478 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:45.341512 20478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:45.348645 20478 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260927 23:01:45.352717 20494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:45.354666 20478 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:01:45.354950 20478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0"
format_stamp: "Formatted at 2026-09-27 23:01:21 on dist-test-slave-5pd3"
server_key: "8b2a4c3beb006b2c055b6e4a6c1cfa11"
server_key_iv: "2467e4934689a2d689bc866987f0f174"
server_key_version: "encryptionkey@0"
I20260927 23:01:45.356225 20478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:45.371161 20478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:45.376097 20478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:45.376423 20478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:45.399137 20478 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:37973
I20260927 23:01:45.399243 20545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:37973 every 8 connection(s)
I20260927 23:01:45.400976 20478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:01:45.403313 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 20478
I20260927 23:01:45.404564 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:44679
--local_ip_for_outbound_sockets=127.4.74.129
--tserver_master_addrs=127.4.74.190:37973
--webserver_port=37369
--webserver_interface=127.4.74.129
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:01:45.415390 20546 sys_catalog.cc:263] Verifying existing consensus state
I20260927 23:01:45.420486 20546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Bootstrap starting.
I20260927 23:01:45.445683 20546 log.cc:824] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:45.462546 20546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 23:01:45.463275 20546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Bootstrap complete.
I20260927 23:01:45.474740 20546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:01:45.476153 20546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106ef2ae2a9a4c328a0d8ab05d269cd0, State: Initialized, Role: FOLLOWER
I20260927 23:01:45.477111 20546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:01:45.477432 20546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:01:45.477586 20546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:01:45.477818 20546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 1 FOLLOWER]: Advancing to term 2
I20260927 23:01:45.483222 20546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:01:45.484215 20546 leader_election.cc:302] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 106ef2ae2a9a4c328a0d8ab05d269cd0; no voters:
I20260927 23:01:45.485119 20546 leader_election.cc:288] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 23:01:45.485302 20550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 23:01:45.486851 20550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 2 LEADER]: Becoming Leader. State: Replica: 106ef2ae2a9a4c328a0d8ab05d269cd0, State: Running, Role: LEADER
I20260927 23:01:45.487756 20550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:01:45.488317 20546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:01:45.491480 20551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } } }
I20260927 23:01:45.491600 20552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 106ef2ae2a9a4c328a0d8ab05d269cd0. Latest consensus state: current_term: 2 leader_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } } }
I20260927 23:01:45.492075 20551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: This master's current role is: LEADER
I20260927 23:01:45.492161 20552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: This master's current role is: LEADER
I20260927 23:01:45.506819 20556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:01:45.517129 20556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:01:45.518934 20556 catalog_manager.cc:1259] Loaded cluster ID: 5e2eae7bdbe048ed992a2bc7427a3d2d
I20260927 23:01:45.519352 20556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:01:45.524168 20556 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:01:45.525985 20556 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Loaded TSK: 0
I20260927 23:01:45.570410 20556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Generated new TSK 1
I20260927 23:01:45.570883 20556 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 23:01:45.794922 20548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:45.795234 20548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:45.795390 20548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:45.803059 20548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:45.803192 20548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:45.803385 20548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:01:45.814581 20548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:44679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=37369
--tserver_master_addrs=127.4.74.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:45.816143 20548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:45.817652 20548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:45.835671 20574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:45.835758 20573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:45.836756 20576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:45.837417 20548 server_base.cc:1034] running on GCE node
I20260927 23:01:45.838344 20548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:45.839846 20548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:45.841140 20548 hybrid_clock.cc:648] HybridClock initialized: now 1790550105841080 us; error 58 us; skew 500 ppm
I20260927 23:01:45.841599 20548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:45 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550105, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:45.853892 20548 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:01:45.856249 20548 webserver.cc:460] Webserver started at http://127.4.74.129:37369/ using document root <none> and password file <none>
I20260927 23:01:45.857259 20548 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:45.857451 20548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:45.866072 20548 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.003s
I20260927 23:01:45.871346 20583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:45.873584 20548 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260927 23:01:45.873848 20548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "17e750d3cd76428b90c8b2f01ac81ee1"
format_stamp: "Formatted at 2026-09-27 23:01:22 on dist-test-slave-5pd3"
server_key: "80be0a2e76b09548ff891c78f5861fc3"
server_key_iv: "7fabb5123cfaf4404508f0511f1f322e"
server_key_version: "encryptionkey@0"
I20260927 23:01:45.875540 20548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:45.899102 20548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:45.904834 20548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:45.905157 20548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:45.906337 20548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:45.908162 20548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:45.908255 20548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:45.908394 20548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:45.908488 20548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:45.955261 20548 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:44679
I20260927 23:01:45.955288 20695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:44679 every 8 connection(s)
I20260927 23:01:45.957082 20548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:01:45.961826 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 20548
I20260927 23:01:45.962874 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:43463
--local_ip_for_outbound_sockets=127.4.74.130
--tserver_master_addrs=127.4.74.190:37973
--webserver_port=45233
--webserver_interface=127.4.74.130
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 23:01:45 dist-test-slave-5pd3 krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550105, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:01:45.999630 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:45.961268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:52159)
0927 23:01:45.962396 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:45.962471 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:45.967055 (+ 4584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:45.970306 (+ 3251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:45.970319 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:45.971235 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:01:45.972229 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:45.972260 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:45.975877 (+ 3617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:45.975887 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:45.978515 (+ 2628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:45.978549 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:45.980765 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:45.980775 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:45.981173 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:45.982276 (+ 1103us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:01:45.982314 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:01:45.986531 (+ 4217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:01:45.992629 (+ 6098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:45.992643 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:45.992667 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:45.993710 (+ 1043us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:45.994445 (+ 735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:45.994452 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:45.994456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:45.994777 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:45.996211 (+ 1434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:01:45.996223 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:01:45.996691 (+ 468us) client_negotiation.cc:769] Sending connection context
0927 23:01:45.997038 (+ 347us) client_negotiation.cc:240] Negotiation successful
0927 23:01:45.997864 (+ 826us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":474,"threads_started":1}
I20260927 23:01:45.999637 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:45.960704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:52159 (local address 127.4.74.190:37973)
0927 23:01:45.963961 (+ 3257us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:45.963985 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:45.964062 (+ 77us) server_negotiation.cc:407] Connection header received
0927 23:01:45.968152 (+ 4090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:45.968172 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:45.968788 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:45.969273 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:45.972743 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:45.975482 (+ 2739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:45.978870 (+ 3388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:45.980345 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:45.980631 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:45.987111 (+ 6480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:45.987127 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:45.987157 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:45.987232 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:45.992308 (+ 5076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:45.993969 (+ 1661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:45.993980 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:45.993988 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:45.994235 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:45.994973 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:45.994978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:45.994980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:45.995964 (+ 984us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:45.996218 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:45.997339 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0927 23:01:45.997859 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2912,"thread_start_us":2667,"threads_started":1}
I20260927 23:01:46.004155 20696 heartbeater.cc:347] Connected to a master server at 127.4.74.190:37973
I20260927 23:01:46.004585 20696 heartbeater.cc:464] Registering TS with master...
I20260927 23:01:46.005592 20696 heartbeater.cc:511] Master 127.4.74.190:37973 requested a full tablet report, sending...
I20260927 23:01:46.008877 20511 ts_manager.cc:194] Registered new tserver with Master: 17e750d3cd76428b90c8b2f01ac81ee1 (127.4.74.129:44679)
I20260927 23:01:46.011013 20511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:52159
W20260927 23:01:46.330684 20700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:46.331022 20700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:46.331121 20700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:46.338824 20700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:46.338965 20700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:46.339090 20700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:01:46.349382 20700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:43463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=45233
--tserver_master_addrs=127.4.74.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:46.350736 20700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:46.352116 20700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:46.368183 20708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:46.368232 20705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:46.368251 20706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:46.370030 20700 server_base.cc:1034] running on GCE node
I20260927 23:01:46.370944 20700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:46.372314 20700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:46.373595 20700 hybrid_clock.cc:648] HybridClock initialized: now 1790550106373505 us; error 73 us; skew 500 ppm
I20260927 23:01:46.374020 20700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:46 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550106, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:46.377777 20700 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:01:46.379652 20700 webserver.cc:460] Webserver started at http://127.4.74.130:45233/ using document root <none> and password file <none>
I20260927 23:01:46.380369 20700 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:46.380496 20700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:46.386570 20700 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260927 23:01:46.389982 20715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:46.391767 20700 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 23:01:46.391971 20700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "0afe24ae78594a2c8abf62313638fc1e"
format_stamp: "Formatted at 2026-09-27 23:01:23 on dist-test-slave-5pd3"
server_key: "6c98633743b5ccd3d22bdae25001af31"
server_key_iv: "7f49210dc934f3be90b62245781ad7c5"
server_key_version: "encryptionkey@0"
I20260927 23:01:46.393024 20700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:46.414582 20700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:46.419601 20700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:46.419925 20700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:46.421331 20700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:46.422999 20700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:46.423149 20700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:46.423305 20700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:46.423398 20700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:46.462102 20700 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:43463
I20260927 23:01:46.462162 20827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:43463 every 8 connection(s)
I20260927 23:01:46.463598 20700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:01:46.465683 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 20700
I20260927 23:01:46.466562 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:36535
--local_ip_for_outbound_sockets=127.4.74.131
--tserver_master_addrs=127.4.74.190:37973
--webserver_port=42053
--webserver_interface=127.4.74.131
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 23:01:46 dist-test-slave-5pd3 krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550106, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:01:46.495332 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:46.467390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:50437 (local address 127.4.74.190:37973)
0927 23:01:46.467693 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:46.467701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:46.469272 (+ 1571us) server_negotiation.cc:407] Connection header received
0927 23:01:46.472764 (+ 3492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:46.472770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:46.472890 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:46.473034 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:46.476147 (+ 3113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:46.477028 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:46.479835 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:46.480643 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:46.480839 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:46.486409 (+ 5570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:46.486418 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:46.486426 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:46.486486 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:46.491082 (+ 4596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:46.492313 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:46.492319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:46.492323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:46.492504 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:46.493076 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:46.493081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:46.493085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:46.493517 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:46.493733 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:46.494594 (+ 861us) server_negotiation.cc:299] Negotiation successful
0927 23:01:46.494907 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260927 23:01:46.496745 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:46.467894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:50437)
0927 23:01:46.468970 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:46.469087 (+ 117us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:46.472231 (+ 3144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:46.473451 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:46.473471 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:46.474506 (+ 1035us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:01:46.475830 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:46.475876 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:46.477289 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:46.477299 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:46.479470 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:46.479495 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:46.481237 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:46.481248 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:46.481714 (+ 466us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:46.482894 (+ 1180us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:01:46.482939 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:01:46.485991 (+ 3052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:01:46.491357 (+ 5366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:46.491379 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:46.491408 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:46.492129 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:46.492635 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:46.492649 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:46.492654 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:46.492925 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:46.493673 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:01:46.493686 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:01:46.494172 (+ 486us) client_negotiation.cc:769] Sending connection context
0927 23:01:46.494500 (+ 328us) client_negotiation.cc:240] Negotiation successful
0927 23:01:46.495199 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":374,"threads_started":1}
I20260927 23:01:46.498880 20828 heartbeater.cc:347] Connected to a master server at 127.4.74.190:37973
I20260927 23:01:46.499297 20828 heartbeater.cc:464] Registering TS with master...
I20260927 23:01:46.500492 20828 heartbeater.cc:511] Master 127.4.74.190:37973 requested a full tablet report, sending...
I20260927 23:01:46.502396 20511 ts_manager.cc:194] Registered new tserver with Master: 0afe24ae78594a2c8abf62313638fc1e (127.4.74.130:43463)
I20260927 23:01:46.503376 20511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:50437
W20260927 23:01:46.830160 20831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:01:46.830546 20831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:01:46.830729 20831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:01:46.838315 20831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:01:46.838510 20831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:01:46.838641 20831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:01:46.848861 20831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:36535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=42053
--tserver_master_addrs=127.4.74.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:01:46.850226 20831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:01:46.851511 20831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:01:46.867211 20839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:46.867202 20836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:01:46.867066 20837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:01:46.868294 20831 server_base.cc:1034] running on GCE node
I20260927 23:01:46.869122 20831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:01:46.870443 20831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:01:46.871712 20831 hybrid_clock.cc:648] HybridClock initialized: now 1790550106871626 us; error 81 us; skew 500 ppm
I20260927 23:01:46.872116 20831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:01:46 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550106, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:01:46.876336 20831 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:01:46.878614 20831 webserver.cc:460] Webserver started at http://127.4.74.131:42053/ using document root <none> and password file <none>
I20260927 23:01:46.879645 20831 fs_manager.cc:359] Metadata directory not provided
I20260927 23:01:46.879839 20831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:01:46.887229 20831 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.002s
I20260927 23:01:46.891697 20846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:46.893762 20831 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:01:46.894100 20831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "058709d6b21d4655b8d9708507719c26"
format_stamp: "Formatted at 2026-09-27 23:01:23 on dist-test-slave-5pd3"
server_key: "36b14638323b01919ef39505faed0496"
server_key_iv: "eeb1af63a65fc3abe5f0ff43e878e696"
server_key_version: "encryptionkey@0"
I20260927 23:01:46.895519 20831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:01:46.918576 20831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:01:46.924336 20831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:01:46.924710 20831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:01:46.926050 20831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:01:46.927857 20831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:01:46.927994 20831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:46.928165 20831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:01:46.928280 20831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:01:46.973088 20831 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:36535
I20260927 23:01:46.973155 20958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:36535 every 8 connection(s)
I20260927 23:01:46.975091 20831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:01:46.982029 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 20831
Sep 27 23:01:46 dist-test-slave-5pd3 krb5kdc[19925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550106, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:01:47.004424 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:46.983271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.4.74.190:37973)
0927 23:01:46.986195 (+ 2924us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:46.986201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:46.986226 (+ 25us) server_negotiation.cc:407] Connection header received
0927 23:01:46.986297 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:46.986301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:46.986383 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:46.986556 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:46.988708 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:46.989764 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:46.991268 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:46.991860 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:46.992011 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:46.995717 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:46.995723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:46.995726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:46.995784 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:47.000155 (+ 4371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.001433 (+ 1278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.001446 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.001450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.001607 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.002304 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.002309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.002313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.002735 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:47.003125 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.003581 (+ 456us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.003920 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2773,"thread_start_us":2605,"threads_started":1}
I20260927 23:01:47.004810 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:46.979139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:47035 (local address 127.4.74.190:37973)
0927 23:01:46.979414 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:46.979420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:46.981174 (+ 1754us) server_negotiation.cc:407] Connection header received
0927 23:01:46.983874 (+ 2700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:46.983885 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:46.984024 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:46.984128 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:46.987040 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:46.987929 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:46.989852 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:46.990346 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:46.990504 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:46.994992 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:46.994998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:46.995001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:46.995043 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:46.998806 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.000375 (+ 1569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.000383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.000386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.000639 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.001452 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.001460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.001464 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.002037 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:47.002288 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.003608 (+ 1320us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.004175 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":228,"server-negotiator.queue_time_us":114}
I20260927 23:01:47.005651 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:46.979863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:47035)
0927 23:01:46.980966 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:46.981018 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:46.983367 (+ 2349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:46.984913 (+ 1546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:46.984930 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:46.986104 (+ 1174us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:01:46.986806 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:46.986827 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:46.988161 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:46.988165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:46.989671 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:46.989685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:46.990606 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:46.990617 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:46.990909 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:46.991685 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:01:46.991711 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:01:46.994709 (+ 2998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:01:46.999074 (+ 4365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:46.999091 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:46.999124 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:47.000057 (+ 933us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:47.000854 (+ 797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:01:47.000859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:01:47.000862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:01:47.001236 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:01:47.002309 (+ 1073us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:01:47.002321 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:01:47.002976 (+ 655us) client_negotiation.cc:769] Sending connection context
0927 23:01:47.003359 (+ 383us) client_negotiation.cc:240] Negotiation successful
0927 23:01:47.004069 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":406,"threads_started":1}
I20260927 23:01:47.007927 20959 heartbeater.cc:347] Connected to a master server at 127.4.74.190:37973
I20260927 23:01:47.008347 20959 heartbeater.cc:464] Registering TS with master...
I20260927 23:01:47.009351 20959 heartbeater.cc:511] Master 127.4.74.190:37973 requested a full tablet report, sending...
I20260927 23:01:47.011212 20511 ts_manager.cc:194] Registered new tserver with Master: 058709d6b21d4655b8d9708507719c26 (127.4.74.131:36535)
I20260927 23:01:47.012389 20511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:47035
I20260927 23:01:47.015688 20696 heartbeater.cc:503] Master 127.4.74.190:37973 was elected leader, sending a full tablet report...
I20260927 23:01:47.022119 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:01:47.030236 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.023462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44622 (local address 127.4.74.190:37973)
0927 23:01:47.023722 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.023727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.024261 (+ 534us) server_negotiation.cc:407] Connection header received
0927 23:01:47.024523 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.024527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.024712 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.024913 (+ 201us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:01:47.025406 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.026250 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.028087 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.028561 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.028685 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.029350 (+ 665us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:01:47.029610 (+ 260us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0927 23:01:47.030014 (+ 404us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44622: token expired
Metrics: {"server-negotiator.queue_time_us":127}
W20260927 23:01:47.030426 20698 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44622: token expired
W20260927 23:01:47.031205 20432 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.74.190:37973)
I20260927 23:01:47.046272 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.031875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.4.74.190:37973)
0927 23:01:47.032050 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.032054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.032073 (+ 19us) server_negotiation.cc:407] Connection header received
0927 23:01:47.032233 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.032237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.032317 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.032419 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:47.033967 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.034739 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.036662 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.037127 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.037244 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.040373 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:47.040379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:47.040382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:47.040421 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:47.043213 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.044188 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.044193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.044196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.044347 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.044789 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.044793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.044795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.045181 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:47.045399 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.045802 (+ 403us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.046036 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 23:01:47.050791 20511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:01:47.061175 20511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:01:47.105476 20971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.078813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:43463 (local address 127.0.0.1:48344)
0927 23:01:47.090752 (+ 11939us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.090806 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:47.091044 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:47.095866 (+ 4822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:47.095874 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:47.095904 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:47.096441 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:47.096456 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.098967 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.098975 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:47.101756 (+ 2781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:47.101774 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.104345 (+ 2571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.104352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:47.104627 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.104677 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 23:01:47.104849 (+ 172us) client_negotiation.cc:240] Negotiation successful
0927 23:01:47.105072 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11562,"mutex_wait_us":1,"thread_start_us":17626,"threads_started":1}
I20260927 23:01:47.105609 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.079048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48344 (local address 127.4.74.130:43463)
0927 23:01:47.094944 (+ 15896us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.094955 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.094991 (+ 36us) server_negotiation.cc:407] Connection header received
0927 23:01:47.095121 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.095133 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.095448 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.096737 (+ 1289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:47.096971 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.098789 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.102002 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.103917 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.104113 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.104410 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.104943 (+ 533us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.105180 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15711,"thread_start_us":15579,"threads_started":1}
I20260927 23:01:47.106791 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.079060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.0.0.1:46850)
0927 23:01:47.096765 (+ 17705us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.096791 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:47.096910 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:47.097791 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:47.097796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:47.097829 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:47.098269 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:47.098286 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.100117 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.100124 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:47.102382 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:47.102398 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.105925 (+ 3527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.105931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:47.106128 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.106165 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 23:01:47.106298 (+ 133us) client_negotiation.cc:240] Negotiation successful
0927 23:01:47.106473 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17559,"thread_start_us":11171,"threads_started":1}
I20260927 23:01:47.107234 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.079418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46850 (local address 127.4.74.129:44679)
0927 23:01:47.094603 (+ 15185us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.094615 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.096952 (+ 2337us) server_negotiation.cc:407] Connection header received
0927 23:01:47.097148 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.097163 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.097491 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.097778 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:47.098518 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.099969 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.102655 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.103959 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.104153 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.104500 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.106529 (+ 2029us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.106810 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14956,"thread_start_us":14758,"threads_started":1}
I20260927 23:01:47.108167 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.094682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:36535 (local address 127.0.0.1:45304)
0927 23:01:47.097526 (+ 2844us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.097544 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:47.097675 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:47.098428 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:47.098434 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:47.098465 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:47.098871 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:47.098885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.100917 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.101380 (+ 463us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:47.104360 (+ 2980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:47.104383 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.107162 (+ 2779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.107170 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:47.107437 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.107485 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 23:01:47.107626 (+ 141us) client_negotiation.cc:240] Negotiation successful
0927 23:01:47.107801 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2711,"thread_start_us":2465,"threads_started":1}
I20260927 23:01:47.108489 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.092303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45304 (local address 127.4.74.131:36535)
0927 23:01:47.094663 (+ 2360us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.094674 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.097678 (+ 3004us) server_negotiation.cc:407] Connection header received
0927 23:01:47.097922 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.097932 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.098198 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.098416 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:47.099075 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.100719 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.104624 (+ 3905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.106519 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.106672 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.106942 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.107838 (+ 896us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.108089 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2151,"thread_start_us":1925,"threads_started":1}
I20260927 23:01:47.112025 20763 tablet_service.cc:1467] Processing CreateTablet for tablet 3a72301fc37041e2a901206782234163 (DEFAULT_TABLE table=test-table [id=4d46e86746744ebe94c04e47185bfa73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:01:47.112397 20631 tablet_service.cc:1467] Processing CreateTablet for tablet 3a72301fc37041e2a901206782234163 (DEFAULT_TABLE table=test-table [id=4d46e86746744ebe94c04e47185bfa73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:01:47.114491 20763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a72301fc37041e2a901206782234163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:01:47.114563 20631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a72301fc37041e2a901206782234163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:01:47.117367 20894 tablet_service.cc:1467] Processing CreateTablet for tablet 3a72301fc37041e2a901206782234163 (DEFAULT_TABLE table=test-table [id=4d46e86746744ebe94c04e47185bfa73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:01:47.119498 20894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a72301fc37041e2a901206782234163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:01:47.129777 20975 tablet_bootstrap.cc:492] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Bootstrap starting.
I20260927 23:01:47.134320 20976 tablet_bootstrap.cc:492] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: Bootstrap starting.
I20260927 23:01:47.135417 20975 tablet_bootstrap.cc:654] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Neither blocks nor log segments found. Creating new log.
I20260927 23:01:47.136993 20975 log.cc:824] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:47.138545 20976 tablet_bootstrap.cc:654] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: Neither blocks nor log segments found. Creating new log.
I20260927 23:01:47.139753 20978 tablet_bootstrap.cc:492] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: Bootstrap starting.
I20260927 23:01:47.140021 20976 log.cc:824] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:47.141455 20975 tablet_bootstrap.cc:492] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: No bootstrap required, opened a new log
I20260927 23:01:47.141889 20975 ts_tablet_manager.cc:1397] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260927 23:01:47.143715 20978 tablet_bootstrap.cc:654] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: Neither blocks nor log segments found. Creating new log.
I20260927 23:01:47.145486 20978 log.cc:824] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: Log is configured to *not* fsync() on all Append() calls
I20260927 23:01:47.147776 20976 tablet_bootstrap.cc:492] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: No bootstrap required, opened a new log
I20260927 23:01:47.148131 20976 ts_tablet_manager.cc:1397] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260927 23:01:47.148974 20978 tablet_bootstrap.cc:492] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: No bootstrap required, opened a new log
I20260927 23:01:47.149290 20978 ts_tablet_manager.cc:1397] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260927 23:01:47.154422 20975 raft_consensus.cc:348] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } } peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } }
I20260927 23:01:47.155023 20975 raft_consensus.cc:374] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:01:47.155191 20975 raft_consensus.cc:729] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e750d3cd76428b90c8b2f01ac81ee1, State: Initialized, Role: FOLLOWER
I20260927 23:01:47.156052 20975 consensus_queue.cc:260] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } } peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } }
I20260927 23:01:47.158329 20975 ts_tablet_manager.cc:1428] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260927 23:01:47.158144 20978 raft_consensus.cc:348] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } } peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } }
I20260927 23:01:47.158823 20978 raft_consensus.cc:374] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:01:47.159008 20978 raft_consensus.cc:729] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058709d6b21d4655b8d9708507719c26, State: Initialized, Role: FOLLOWER
I20260927 23:01:47.160003 20978 consensus_queue.cc:260] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } } peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } }
I20260927 23:01:47.160470 20976 raft_consensus.cc:348] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } } peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } }
I20260927 23:01:47.161882 20959 heartbeater.cc:503] Master 127.4.74.190:37973 was elected leader, sending a full tablet report...
I20260927 23:01:47.161587 20976 raft_consensus.cc:374] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:01:47.162904 20976 raft_consensus.cc:729] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0afe24ae78594a2c8abf62313638fc1e, State: Initialized, Role: FOLLOWER
I20260927 23:01:47.163226 20978 ts_tablet_manager.cc:1428] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260927 23:01:47.163942 20976 consensus_queue.cc:260] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } } peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } }
I20260927 23:01:47.166301 20828 heartbeater.cc:503] Master 127.4.74.190:37973 was elected leader, sending a full tablet report...
I20260927 23:01:47.167066 20976 ts_tablet_manager.cc:1428] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
I20260927 23:01:47.193938 20981 raft_consensus.cc:482] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:01:47.194366 20981 raft_consensus.cc:504] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } } peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } }
I20260927 23:01:47.197300 20981 leader_election.cc:288] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0afe24ae78594a2c8abf62313638fc1e (127.4.74.130:43463), 058709d6b21d4655b8d9708507719c26 (127.4.74.131:36535)
I20260927 23:01:47.207172 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.197979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:46549 (local address 127.4.74.131:36535)
0927 23:01:47.198299 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.198307 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.199477 (+ 1170us) server_negotiation.cc:407] Connection header received
0927 23:01:47.199779 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.199785 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.199919 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.200022 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:47.201375 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.202598 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.205201 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.206042 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.206264 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.206347 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.206716 (+ 369us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.206861 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260927 23:01:47.207356 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.198190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:56571 (local address 127.4.74.130:43463)
0927 23:01:47.198534 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.198545 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.199602 (+ 1057us) server_negotiation.cc:407] Connection header received
0927 23:01:47.199778 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.199784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.199903 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.200022 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:01:47.201407 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.202614 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.205251 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.206159 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.206420 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.206527 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.206864 (+ 337us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.207033 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260927 23:01:47.207499 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.197814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:36535 (local address 127.4.74.129:46549)
0927 23:01:47.199258 (+ 1444us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.199291 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:47.199540 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:47.200132 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:47.200138 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:47.200204 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:47.201063 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:47.201081 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.202905 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.202912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:47.204965 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:47.204986 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.206288 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.206295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:47.206484 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.206554 (+ 70us) client_negotiation.cc:769] Sending connection context
0927 23:01:47.206688 (+ 134us) client_negotiation.cc:240] Negotiation successful
0927 23:01:47.206853 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1233,"mutex_wait_us":147,"spinlock_wait_cycles":37248,"thread_start_us":1013,"threads_started":1}
I20260927 23:01:47.207600 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.197846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:43463 (local address 127.4.74.129:56571)
0927 23:01:47.199289 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.199305 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 23:01:47.199604 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:01:47.200132 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:01:47.200138 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:01:47.200187 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:01:47.201063 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:47.201081 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.202849 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.202860 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:47.204974 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:01:47.204996 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.206459 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.206466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:01:47.206670 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.206698 (+ 28us) client_negotiation.cc:769] Sending connection context
0927 23:01:47.206818 (+ 120us) client_negotiation.cc:240] Negotiation successful
0927 23:01:47.206991 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1327,"mutex_wait_us":143,"spinlock_wait_cycles":33024,"thread_start_us":809,"threads_started":1}
I20260927 23:01:47.208940 20914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a72301fc37041e2a901206782234163" candidate_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058709d6b21d4655b8d9708507719c26" is_pre_election: true
I20260927 23:01:47.209218 20783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a72301fc37041e2a901206782234163" candidate_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0afe24ae78594a2c8abf62313638fc1e" is_pre_election: true
I20260927 23:01:47.209774 20914 raft_consensus.cc:2454] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e750d3cd76428b90c8b2f01ac81ee1 in term 0.
I20260927 23:01:47.209910 20783 raft_consensus.cc:2454] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e750d3cd76428b90c8b2f01ac81ee1 in term 0.
W20260927 23:01:47.210654 20697 tablet.cc:2369] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:01:47.210774 20584 leader_election.cc:302] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 058709d6b21d4655b8d9708507719c26, 17e750d3cd76428b90c8b2f01ac81ee1; no voters:
I20260927 23:01:47.211345 20981 raft_consensus.cc:2781] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:01:47.211521 20981 raft_consensus.cc:482] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:01:47.211676 20981 raft_consensus.cc:3037] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:01:47.214052 20981 raft_consensus.cc:504] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } } peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } }
I20260927 23:01:47.214876 20981 leader_election.cc:288] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [CANDIDATE]: Term 1 election: Requested vote from peers 0afe24ae78594a2c8abf62313638fc1e (127.4.74.130:43463), 058709d6b21d4655b8d9708507719c26 (127.4.74.131:36535)
I20260927 23:01:47.215672 20914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a72301fc37041e2a901206782234163" candidate_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "058709d6b21d4655b8d9708507719c26"
I20260927 23:01:47.215750 20783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a72301fc37041e2a901206782234163" candidate_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0afe24ae78594a2c8abf62313638fc1e"
I20260927 23:01:47.215951 20914 raft_consensus.cc:3037] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:01:47.215997 20783 raft_consensus.cc:3037] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:01:47.215974 20829 tablet.cc:2369] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:01:47.219612 20783 raft_consensus.cc:2454] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e750d3cd76428b90c8b2f01ac81ee1 in term 1.
I20260927 23:01:47.219681 20914 raft_consensus.cc:2454] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e750d3cd76428b90c8b2f01ac81ee1 in term 1.
I20260927 23:01:47.220331 20586 leader_election.cc:302] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0afe24ae78594a2c8abf62313638fc1e, 17e750d3cd76428b90c8b2f01ac81ee1; no voters:
I20260927 23:01:47.220836 20981 raft_consensus.cc:2781] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:01:47.221378 20981 raft_consensus.cc:686] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 1 LEADER]: Becoming Leader. State: Replica: 17e750d3cd76428b90c8b2f01ac81ee1, State: Running, Role: LEADER
I20260927 23:01:47.221927 20981 consensus_queue.cc:237] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } } peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } }
W20260927 23:01:47.227438 20960 tablet.cc:2369] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:01:47.228263 20510 catalog_manager.cc:5495] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e750d3cd76428b90c8b2f01ac81ee1 (127.4.74.129). New cstate: current_term: 1 leader_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17e750d3cd76428b90c8b2f01ac81ee1" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 } health_report { overall_health: UNKNOWN } } }
I20260927 23:01:47.256716 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.249841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.4.74.129:44679)
0927 23:01:47.250145 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.250154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.250285 (+ 131us) server_negotiation.cc:407] Connection header received
0927 23:01:47.250650 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.250654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.250882 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.251121 (+ 239us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:01:47.251638 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.252439 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.253930 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.254395 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.254545 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.255231 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:01:47.256015 (+ 784us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:01:47.256190 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.256274 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.256380 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260927 23:01:47.267180 20783 raft_consensus.cc:1261] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e [term 1 FOLLOWER]: Refusing update from remote peer 17e750d3cd76428b90c8b2f01ac81ee1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:01:47.267177 20914 raft_consensus.cc:1261] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26 [term 1 FOLLOWER]: Refusing update from remote peer 17e750d3cd76428b90c8b2f01ac81ee1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:01:47.268810 20986 consensus_queue.cc:1035] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "058709d6b21d4655b8d9708507719c26" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 36535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:01:47.269450 20981 consensus_queue.cc:1035] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0afe24ae78594a2c8abf62313638fc1e" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:01:47.325613 20510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:01:47.325990 20510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:01:47.331290 20510 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Sending DeleteTablet for 3 replicas of tablet 3a72301fc37041e2a901206782234163
I20260927 23:01:47.333137 20763 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a72301fc37041e2a901206782234163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:01:47 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:48344
I20260927 23:01:47.333310 20894 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a72301fc37041e2a901206782234163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:01:47 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:45304
I20260927 23:01:47.333324 20631 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a72301fc37041e2a901206782234163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:01:47 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:46850
I20260927 23:01:47.335294 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 20548
I20260927 23:01:47.349113 21001 tablet_replica.cc:330] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: stopping tablet replica
I20260927 23:01:47.350241 21001 raft_consensus.cc:2229] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:01:47.351375 20999 tablet_replica.cc:330] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: stopping tablet replica
I20260927 23:01:47.351752 21001 raft_consensus.cc:2258] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:01:47.352801 21000 tablet_replica.cc:330] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: stopping tablet replica
I20260927 23:01:47.353524 21000 raft_consensus.cc:2229] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:01:47.354521 21000 raft_consensus.cc:2258] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:01:47.355033 20999 raft_consensus.cc:2229] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:01:47.355374 21001 ts_tablet_manager.cc:1905] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:01:47.355749 20999 raft_consensus.cc:2258] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:01:47.358357 20999 ts_tablet_manager.cc:1905] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:01:47.358551 21000 ts_tablet_manager.cc:1905] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:01:47.366801 20999 ts_tablet_manager.cc:1918] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:01:47.367086 20999 log.cc:1196] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/3a72301fc37041e2a901206782234163
I20260927 23:01:47.367491 21001 ts_tablet_manager.cc:1918] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:01:47.367807 20999 ts_tablet_manager.cc:1939] T 3a72301fc37041e2a901206782234163 P 0afe24ae78594a2c8abf62313638fc1e: Deleting consensus metadata
I20260927 23:01:47.367990 21001 log.cc:1196] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/3a72301fc37041e2a901206782234163
I20260927 23:01:47.368758 21001 ts_tablet_manager.cc:1939] T 3a72301fc37041e2a901206782234163 P 058709d6b21d4655b8d9708507719c26: Deleting consensus metadata
I20260927 23:01:47.370430 20497 catalog_manager.cc:4841] TS 0afe24ae78594a2c8abf62313638fc1e (127.4.74.130:43463): tablet 3a72301fc37041e2a901206782234163 (table test-table [id=4d46e86746744ebe94c04e47185bfa73]) successfully deleted
I20260927 23:01:47.371260 20495 catalog_manager.cc:4841] TS 058709d6b21d4655b8d9708507719c26 (127.4.74.131:36535): tablet 3a72301fc37041e2a901206782234163 (table test-table [id=4d46e86746744ebe94c04e47185bfa73]) successfully deleted
I20260927 23:01:47.373543 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.337645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46870 (local address 127.4.74.129:44679)
0927 23:01:47.338143 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.338151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.338239 (+ 88us) server_negotiation.cc:407] Connection header received
0927 23:01:47.338521 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.338527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.339043 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.339262 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:47.344862 (+ 5600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.346763 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.350706 (+ 3943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.351594 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.351747 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.360765 (+ 9018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:47.360793 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:47.360815 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:47.360880 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:47.367269 (+ 6389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.368840 (+ 1571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.368856 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.368860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.369132 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.369741 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.369747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.369751 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.371160 (+ 1409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:47.371653 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.372366 (+ 713us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.372758 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260927 23:01:47.375414 20691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:47.500916 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 20548
W20260927 23:01:47.520783 20495 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.74.129:44679 (error 108)
I20260927 23:01:47.521006 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 20700
I20260927 23:01:47.522760 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.522241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.0.0.1:46876)
0927 23:01:47.522442 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.522583 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260927 23:01:47.523252 20495 catalog_manager.cc:4584] TS 17e750d3cd76428b90c8b2f01ac81ee1 (127.4.74.129:44679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3a72301fc37041e2a901206782234163: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44679: connect: Connection refused (error 111)
I20260927 23:01:47.538302 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.521994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48358 (local address 127.4.74.130:43463)
0927 23:01:47.522187 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.522193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.522442 (+ 249us) server_negotiation.cc:407] Connection header received
0927 23:01:47.522663 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.522668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.522770 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.522917 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:47.524852 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.525762 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.527397 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.527979 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.528139 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.531112 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:47.531133 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:47.531155 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:47.531213 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:47.534906 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.536082 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.536091 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.536094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.536289 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.536667 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.536670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.536672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.537150 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:47.537422 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.537711 (+ 289us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.537986 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 23:01:47.539299 20823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:47.577749 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.577090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.0.0.1:46890)
0927 23:01:47.577384 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.577550 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260927 23:01:47.578992 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.578604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.0.0.1:46892)
0927 23:01:47.578768 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.578855 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 23:01:47.639799 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.639166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.0.0.1:46898)
0927 23:01:47.639475 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.639642 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260927 23:01:47.640887 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.640543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.0.0.1:46914)
0927 23:01:47.640688 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.640765 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 23:01:47.650586 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 20700
I20260927 23:01:47.671428 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 20831
I20260927 23:01:47.687902 20973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.672498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45316 (local address 127.4.74.131:36535)
0927 23:01:47.672690 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 23:01:47.672695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:01:47.672719 (+ 24us) server_negotiation.cc:407] Connection header received
0927 23:01:47.672918 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:01:47.672924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:01:47.673002 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:01:47.673112 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:01:47.674933 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.675870 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.677393 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:01:47.677826 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:01:47.677947 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:01:47.681198 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:01:47.681217 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:01:47.681237 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:01:47.681280 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:01:47.684523 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.685463 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.685471 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.685473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.685651 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:01:47.686028 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:01:47.686032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:01:47.686034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:01:47.686439 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:01:47.686833 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0927 23:01:47.687139 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 23:01:47.687338 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260927 23:01:47.688750 20954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:47.721890 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.721197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.0.0.1:46918)
0927 23:01:47.721502 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.721651 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260927 23:01:47.723206 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.722745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.0.0.1:46926)
0927 23:01:47.722983 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.723079 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260927 23:01:47.787240 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 20831
I20260927 23:01:47.806401 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 20478
I20260927 23:01:47.807947 20541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:01:47.934558 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:01:47.933687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44679 (local address 127.0.0.1:46934)
0927 23:01:47.933970 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0927 23:01:47.934149 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260927 23:01:47.935302 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 20478
I20260927 23:02:07.960453 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:37973
--webserver_interface=127.4.74.190
--webserver_port=38965
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:37973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:08.338794 21007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:08.339133 21007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:08.339228 21007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:08.346912 21007 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 23:02:08.347039 21007 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 23:02:08.347079 21007 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 23:02:08.347124 21007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:08.347173 21007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:08.347250 21007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:08.347302 21007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:08.347352 21007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:08.357914 21007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:37973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:37973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=38965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:08.359328 21007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:08.360823 21007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:08.375048 21012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:08.375370 21013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:08.375897 21015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:08.376255 21007 server_base.cc:1034] running on GCE node
I20260927 23:02:08.377134 21007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:08.378728 21007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:08.380013 21007 hybrid_clock.cc:648] HybridClock initialized: now 1790550128379926 us; error 61 us; skew 500 ppm
I20260927 23:02:08.380416 21007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:08 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550128, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:08.392738 21007 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:02:08.395133 21007 webserver.cc:460] Webserver started at http://127.4.74.190:38965/ using document root <none> and password file <none>
I20260927 23:02:08.396075 21007 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:08.396232 21007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:08.403047 21007 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260927 23:02:08.407140 21022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:08.409235 21007 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260927 23:02:08.409514 21007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0"
format_stamp: "Formatted at 2026-09-27 23:01:21 on dist-test-slave-5pd3"
server_key: "8b2a4c3beb006b2c055b6e4a6c1cfa11"
server_key_iv: "2467e4934689a2d689bc866987f0f174"
server_key_version: "encryptionkey@0"
I20260927 23:02:08.410679 21007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:08.438788 21007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:08.443276 21007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:08.443607 21007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:08.465839 21007 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:37973
I20260927 23:02:08.465893 21073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:37973 every 8 connection(s)
I20260927 23:02:08.467525 21007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:08.474285 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 21007
I20260927 23:02:08.475446 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:44679
--local_ip_for_outbound_sockets=127.4.74.129
--tserver_master_addrs=127.4.74.190:37973
--webserver_port=37369
--webserver_interface=127.4.74.129
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:02:08.477385 21074 sys_catalog.cc:263] Verifying existing consensus state
I20260927 23:02:08.483649 21074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Bootstrap starting.
I20260927 23:02:08.510455 21074 log.cc:824] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:08.532505 21074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 23:02:08.533282 21074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Bootstrap complete.
I20260927 23:02:08.545260 21074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:02:08.546489 21074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106ef2ae2a9a4c328a0d8ab05d269cd0, State: Initialized, Role: FOLLOWER
I20260927 23:02:08.547416 21074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:02:08.547780 21074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:08.547917 21074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:08.548115 21074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 2 FOLLOWER]: Advancing to term 3
I20260927 23:02:08.553323 21074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:02:08.554096 21074 leader_election.cc:302] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 106ef2ae2a9a4c328a0d8ab05d269cd0; no voters:
I20260927 23:02:08.554999 21074 leader_election.cc:288] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 23:02:08.555176 21078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 23:02:08.556627 21078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [term 3 LEADER]: Becoming Leader. State: Replica: 106ef2ae2a9a4c328a0d8ab05d269cd0, State: Running, Role: LEADER
I20260927 23:02:08.557473 21078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } }
I20260927 23:02:08.557971 21074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:08.560819 21080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 106ef2ae2a9a4c328a0d8ab05d269cd0. Latest consensus state: current_term: 3 leader_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } } }
I20260927 23:02:08.561439 21080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:08.561106 21079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106ef2ae2a9a4c328a0d8ab05d269cd0" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37973 } } }
I20260927 23:02:08.561636 21079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:08.573166 21085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:08.587723 21085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:08.590698 21085 catalog_manager.cc:1259] Loaded cluster ID: 5e2eae7bdbe048ed992a2bc7427a3d2d
I20260927 23:02:08.590945 21085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:08.597328 21085 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:08.599391 21085 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Loaded TSK: 0
I20260927 23:02:08.599588 21085 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 106ef2ae2a9a4c328a0d8ab05d269cd0: Loaded TSK: 1
F20260927 23:02:08.600792 21085 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f09af2279ad google::LogMessage::Fail() at ??:0
@ 0x7f09af22a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f09af2274de google::LogMessage::Flush() at ??:0
@ 0x7f09af228a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f09b2f8b7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f09b2f7a73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f09c4c561c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f09c4c55783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f09c4ce12a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f09c4ce10c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f09c4d2b5e9 std::function<>::operator()() at ??:0
@ 0x7f09c4c59cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f09c4c57c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f09c4cdf610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f09c4cdf439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f09b1e73345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f09b1e7691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f09b1e76799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f09b1e51eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f09b512b6db start_thread at ??:0
@ 0x7f09ae0c161f clone at ??:0
W20260927 23:02:08.931159 21076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:08.931476 21076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:08.931649 21076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:08.939318 21076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:08.939451 21076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:08.939679 21076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:02:08.950763 21076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:44679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=37369
--tserver_master_addrs=127.4.74.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:08.952291 21076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:08.953657 21076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:08.970628 21121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:08.970853 21124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:08.971537 21122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:08.973434 21076 server_base.cc:1034] running on GCE node
I20260927 23:02:08.974632 21076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:08.975983 21076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:08.977262 21076 hybrid_clock.cc:648] HybridClock initialized: now 1790550128977199 us; error 65 us; skew 500 ppm
I20260927 23:02:08.977671 21076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:08 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550128, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:08.981467 21076 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:02:08.983641 21076 webserver.cc:460] Webserver started at http://127.4.74.129:37369/ using document root <none> and password file <none>
I20260927 23:02:08.984479 21076 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:08.984650 21076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:08.991403 21076 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260927 23:02:08.995198 21131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:08.997059 21076 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:02:08.997308 21076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "17e750d3cd76428b90c8b2f01ac81ee1"
format_stamp: "Formatted at 2026-09-27 23:01:22 on dist-test-slave-5pd3"
server_key: "80be0a2e76b09548ff891c78f5861fc3"
server_key_iv: "7fabb5123cfaf4404508f0511f1f322e"
server_key_version: "encryptionkey@0"
I20260927 23:02:08.998543 21076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:09.022974 21076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:09.028359 21076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:09.028720 21076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:09.030190 21076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:09.032387 21138 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260927 23:02:09.039182 21138 ts_tablet_manager.cc:1852] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260927 23:02:09.040504 21138 ts_tablet_manager.cc:1905] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:09.049834 21138 ts_tablet_manager.cc:1918] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:09.050132 21138 log.cc:1196] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/3a72301fc37041e2a901206782234163
I20260927 23:02:09.050733 21138 ts_tablet_manager.cc:1939] T 3a72301fc37041e2a901206782234163 P 17e750d3cd76428b90c8b2f01ac81ee1: Deleting consensus metadata
I20260927 23:02:09.051301 21076 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260927 23:02:09.051535 21076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.020s user 0.001s sys 0.000s
I20260927 23:02:09.051785 21076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:09.051905 21076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:09.093854 21076 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:44679
I20260927 23:02:09.093964 21244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:44679 every 8 connection(s)
I20260927 23:02:09.095395 21076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:02:09.102949 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 21076
I20260927 23:02:09.103114 21247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:09.099884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:39115)
0927 23:02:09.100980 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:09.101292 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":404,"threads_started":1}
I20260927 23:02:09.103914 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:43463
--local_ip_for_outbound_sockets=127.4.74.130
--tserver_master_addrs=127.4.74.190:37973
--webserver_port=45233
--webserver_interface=127.4.74.130
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:09.104130 21245 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:37973 (0 consecutive failures): Network error: Failed to ping master at 127.4.74.190:37973: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
I20260927 23:02:09.105438 21247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:09.104784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:44007)
0927 23:02:09.105071 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:09.105221 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260927 23:02:09.106822 21247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:09.106199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:38745)
0927 23:02:09.106415 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:09.106542 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260927 23:02:09.107225 21245 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 23:02:09.454633 21248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:09.455003 21248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:09.455170 21248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:09.462837 21248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:09.463023 21248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:09.463268 21248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:02:09.473469 21248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:43463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=45233
--tserver_master_addrs=127.4.74.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:09.474869 21248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:09.476181 21248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:09.491472 21254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:09.491497 21253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:09.491374 21256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:09.493096 21248 server_base.cc:1034] running on GCE node
I20260927 23:02:09.493861 21248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:09.494982 21248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:09.496248 21248 hybrid_clock.cc:648] HybridClock initialized: now 1790550129496187 us; error 49 us; skew 500 ppm
I20260927 23:02:09.496616 21248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:09 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550129, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:09.500360 21248 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:02:09.502107 21248 webserver.cc:460] Webserver started at http://127.4.74.130:45233/ using document root <none> and password file <none>
I20260927 23:02:09.502825 21248 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:09.502938 21248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:09.509022 21248 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:02:09.512486 21263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:09.514313 21248 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:02:09.514559 21248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "0afe24ae78594a2c8abf62313638fc1e"
format_stamp: "Formatted at 2026-09-27 23:01:23 on dist-test-slave-5pd3"
server_key: "6c98633743b5ccd3d22bdae25001af31"
server_key_iv: "7f49210dc934f3be90b62245781ad7c5"
server_key_version: "encryptionkey@0"
I20260927 23:02:09.515827 21248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:09.540071 21248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:09.544723 21248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:09.545024 21248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:09.546181 21248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:09.547900 21248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:09.548097 21248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:09.548247 21248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:09.548343 21248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:09.589926 21248 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:43463
I20260927 23:02:09.590054 21375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:43463 every 8 connection(s)
I20260927 23:02:09.592137 21248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:02:09.596136 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 21248
I20260927 23:02:09.597127 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:36535
--local_ip_for_outbound_sockets=127.4.74.131
--tserver_master_addrs=127.4.74.190:37973
--webserver_port=42053
--webserver_interface=127.4.74.131
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:02:09.599334 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:09.595951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:36567)
0927 23:02:09.597265 (+ 1314us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:09.597638 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":514,"threads_started":1}
W20260927 23:02:09.600389 21376 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:37973 (0 consecutive failures): Network error: Failed to ping master at 127.4.74.190:37973: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
I20260927 23:02:09.601675 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:09.601070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:53967)
0927 23:02:09.601331 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:09.601470 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260927 23:02:09.603204 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:09.602593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:45663)
0927 23:02:09.602848 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:09.603005 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260927 23:02:09.603605 21376 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 23:02:09.954484 21379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:09.954859 21379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:09.955016 21379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:09.962730 21379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:09.962920 21379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:09.963095 21379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:02:09.973738 21379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:36535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=42053
--tserver_master_addrs=127.4.74.190:37973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:09.975409 21379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:09.977018 21379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:09.992326 21385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:09.992686 21384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:09.993031 21387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:09.993706 21379 server_base.cc:1034] running on GCE node
I20260927 23:02:09.995009 21379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:09.997037 21379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:09.998378 21379 hybrid_clock.cc:648] HybridClock initialized: now 1790550129998292 us; error 92 us; skew 500 ppm
I20260927 23:02:09.998988 21379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:10 dist-test-slave-5pd3 krb5kdc[19925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550130, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:10.009213 21379 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:02:10.012040 21379 webserver.cc:460] Webserver started at http://127.4.74.131:42053/ using document root <none> and password file <none>
I20260927 23:02:10.012910 21379 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:10.013072 21379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:10.025641 21379 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.004s
I20260927 23:02:10.030164 21394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:10.032553 21379 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260927 23:02:10.032856 21379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "058709d6b21d4655b8d9708507719c26"
format_stamp: "Formatted at 2026-09-27 23:01:23 on dist-test-slave-5pd3"
server_key: "36b14638323b01919ef39505faed0496"
server_key_iv: "eeb1af63a65fc3abe5f0ff43e878e696"
server_key_version: "encryptionkey@0"
I20260927 23:02:10.034021 21379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:10.051330 21379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:10.057451 21379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:10.058022 21379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:10.059439 21379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:10.061172 21379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:10.061313 21379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:10.061450 21379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:10.061537 21379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:10.109786 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:10.108275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:41255)
0927 23:02:10.109105 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:10.109293 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":337,"threads_started":1}
I20260927 23:02:10.119359 21379 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:36535
I20260927 23:02:10.119679 21507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:36535 every 8 connection(s)
I20260927 23:02:10.121143 21379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:02:10.127748 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:10.125113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:45681)
0927 23:02:10.126099 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:10.126434 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":392,"threads_started":1}
W20260927 23:02:10.128767 21508 heartbeater.cc:644] Failed to heartbeat to 127.4.74.190:37973 (0 consecutive failures): Network error: Failed to ping master at 127.4.74.190:37973: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
I20260927 23:02:10.130403 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:10.129725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:52263)
0927 23:02:10.130058 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:10.130195 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260927 23:02:10.130470 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 21379
I20260927 23:02:10.132097 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:10.131419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:60537)
0927 23:02:10.131819 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:10.131917 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255}
W20260927 23:02:10.132524 21508 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 23:02:10.605608 21512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:10.604502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:42435)
0927 23:02:10.605162 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:10.605288 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":332,"threads_started":1}
I20260927 23:02:11.112021 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:11.111060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:33303)
0927 23:02:11.111736 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:11.111843 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":352,"threads_started":1}
I20260927 23:02:11.134286 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:11.133241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:44871)
0927 23:02:11.133880 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:11.134012 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":287,"threads_started":1}
I20260927 23:02:11.607683 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:11.606656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:37209)
0927 23:02:11.607289 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:11.607424 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":303,"threads_started":1}
I20260927 23:02:12.114231 21516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:12.113232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:60127)
0927 23:02:12.113882 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:12.113999 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":313,"threads_started":1}
I20260927 23:02:12.136662 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:12.135610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:55063)
0927 23:02:12.136317 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:12.136423 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":385,"threads_started":1}
I20260927 23:02:12.609834 21518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:12.608863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:34731)
0927 23:02:12.609462 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:12.609583 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":313,"threads_started":1}
I20260927 23:02:13.116482 21519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:13.115488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:49533)
0927 23:02:13.116194 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:13.116304 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":312,"threads_started":1}
I20260927 23:02:13.138937 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:13.137871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:53673)
0927 23:02:13.138551 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:13.138665 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":302,"threads_started":1}
I20260927 23:02:13.612157 21521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:13.611026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:55141)
0927 23:02:13.611787 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:13.611920 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":325,"threads_started":1}
I20260927 23:02:14.118675 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:14.117678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:57139)
0927 23:02:14.118319 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:14.118430 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":299,"threads_started":1}
I20260927 23:02:14.141179 21523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:14.140274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:40135)
0927 23:02:14.140923 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:14.141000 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":328,"threads_started":1}
I20260927 23:02:14.614584 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:14.613596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:55911)
0927 23:02:14.614253 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:14.614378 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":336,"threads_started":1}
I20260927 23:02:15.121229 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:15.120101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:49007)
0927 23:02:15.120848 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:15.121002 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":380,"threads_started":1}
I20260927 23:02:15.143384 21526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:15.142475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:53723)
0927 23:02:15.143104 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:15.143208 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":373,"threads_started":1}
I20260927 23:02:15.617074 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:15.615995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:42715)
0927 23:02:15.616675 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:15.616848 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":370,"threads_started":1}
I20260927 23:02:16.123438 21528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:16.122306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:35143)
0927 23:02:16.123045 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:16.123192 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":302,"threads_started":1}
I20260927 23:02:16.145502 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:16.144517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:56265)
0927 23:02:16.145182 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:16.145302 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":339,"threads_started":1}
I20260927 23:02:16.619844 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:16.618498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:52581)
0927 23:02:16.619334 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:16.619473 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":341,"threads_started":1}
I20260927 23:02:17.125581 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:17.124645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:48079)
0927 23:02:17.125271 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:17.125384 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":308,"threads_started":1}
I20260927 23:02:17.147626 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:17.146561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:35505)
0927 23:02:17.147208 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:17.147317 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":287,"threads_started":1}
I20260927 23:02:17.622134 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:17.621146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:38699)
0927 23:02:17.621762 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:17.621882 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":315,"threads_started":1}
I20260927 23:02:18.128333 21534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:18.127036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:38183)
0927 23:02:18.127927 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:18.128086 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":337,"threads_started":1}
I20260927 23:02:18.149850 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:18.148918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:57413)
0927 23:02:18.149550 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:18.149656 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":367,"threads_started":1}
I20260927 23:02:18.624475 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:18.623442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:34053)
0927 23:02:18.624144 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:18.624272 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":384,"threads_started":1}
I20260927 23:02:19.130764 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:19.129743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:35497)
0927 23:02:19.130443 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:19.130558 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":337,"threads_started":1}
I20260927 23:02:19.152201 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:19.151099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:60353)
0927 23:02:19.151844 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:19.151962 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":316,"threads_started":1}
I20260927 23:02:19.626659 21539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:19.625701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:56269)
0927 23:02:19.626347 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:19.626463 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":344,"threads_started":1}
I20260927 23:02:20.133193 21540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:20.132209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:36809)
0927 23:02:20.132861 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:20.132971 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":368,"threads_started":1}
I20260927 23:02:20.154428 21541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:20.153392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:45057)
0927 23:02:20.154089 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:20.154213 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":371,"threads_started":1}
I20260927 23:02:20.628871 21542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:20.627945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:34869)
0927 23:02:20.628554 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:20.628671 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":321,"threads_started":1}
I20260927 23:02:21.135439 21543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:21.134439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:46941)
0927 23:02:21.135097 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:21.135221 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":333,"threads_started":1}
I20260927 23:02:21.156529 21544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:21.155674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:46869)
0927 23:02:21.156227 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:21.156318 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":304,"threads_started":1}
I20260927 23:02:21.631052 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:21.630180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:41517)
0927 23:02:21.630750 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:21.630884 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":306,"threads_started":1}
I20260927 23:02:22.137650 21546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:22.136575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:43045)
0927 23:02:22.137226 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:22.137392 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":317,"threads_started":1}
I20260927 23:02:22.158547 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:22.157610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:49711)
0927 23:02:22.158244 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:22.158359 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":327,"threads_started":1}
I20260927 23:02:22.633237 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:22.632273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:49399)
0927 23:02:22.632912 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:22.633019 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":341,"threads_started":1}
I20260927 23:02:23.140115 21549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:23.138934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:55827)
0927 23:02:23.139768 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:23.139887 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":381,"threads_started":1}
I20260927 23:02:23.161144 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:23.160021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:33615)
0927 23:02:23.160796 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:23.160912 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":424,"threads_started":1}
I20260927 23:02:23.635401 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:23.634481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:42051)
0927 23:02:23.635099 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:23.635190 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":317,"threads_started":1}
I20260927 23:02:24.142812 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:24.141746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:57779)
0927 23:02:24.142411 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:24.142558 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":310,"threads_started":1}
I20260927 23:02:24.163338 21553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:24.162325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:42309)
0927 23:02:24.162941 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:24.163074 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":317,"threads_started":1}
I20260927 23:02:24.638304 21554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:24.637149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.130:54357)
0927 23:02:24.637925 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:24.638083 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":454,"threads_started":1}
I20260927 23:02:25.133729 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 21076
I20260927 23:02:25.145479 21556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:25.144160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.129:34199)
0927 23:02:25.144826 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:25.145013 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":328,"threads_started":1}
I20260927 23:02:25.157871 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:25.135088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.4.74.129:44679)
0927 23:02:25.135931 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:25.135959 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:25.136023 (+ 64us) server_negotiation.cc:407] Connection header received
0927 23:02:25.138329 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:25.138344 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:25.139041 (+ 697us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:25.139483 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:25.141562 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:25.143149 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:25.145183 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:25.146457 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:25.146659 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:25.149831 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:25.149841 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:25.149865 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:25.149961 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:25.153616 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:25.154541 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:25.154548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:25.154556 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:25.154762 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:25.155217 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:25.155221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:25.155223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:25.156045 (+ 822us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:25.156359 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:25.156918 (+ 559us) server_negotiation.cc:299] Negotiation successful
0927 23:02:25.157418 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":389,"threads_started":1}
I20260927 23:02:25.159857 21240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:25.165735 21558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:25.164830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37973 (local address 127.4.74.131:46661)
0927 23:02:25.165393 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:25.165533 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.190:37973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":348,"threads_started":1}
I20260927 23:02:25.260427 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 21076
I20260927 23:02:25.279116 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 21248
I20260927 23:02:25.299144 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:25.280307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.4.74.130:43463)
0927 23:02:25.280986 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:25.281001 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:25.281040 (+ 39us) server_negotiation.cc:407] Connection header received
0927 23:02:25.282719 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:25.282730 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:25.283176 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:25.283545 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:25.285105 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:25.286371 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:25.287987 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:25.288679 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:25.288911 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:25.291695 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:25.291703 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:25.291720 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:25.291762 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:25.294872 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:25.295897 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:25.295908 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:25.295918 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:25.296225 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:25.296643 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:25.296647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:25.296649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:25.297367 (+ 718us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:25.297662 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:25.298295 (+ 633us) server_negotiation.cc:299] Negotiation successful
0927 23:02:25.298783 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":303,"threads_started":1}
I20260927 23:02:25.300805 21371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:25.392536 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 21248
I20260927 23:02:25.413713 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 21379
I20260927 23:02:25.433595 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:25.415105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35896 (local address 127.4.74.131:36535)
0927 23:02:25.415975 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:25.415991 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:25.416035 (+ 44us) server_negotiation.cc:407] Connection header received
0927 23:02:25.417775 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:25.417785 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:25.418225 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:25.418587 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:25.420190 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:25.421470 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:25.422776 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:25.423448 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:25.423681 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:25.426638 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:25.426646 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:25.426663 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:25.426715 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:25.429892 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:25.430670 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:25.430677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:25.430684 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:25.430858 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:25.431242 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:25.431247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:25.431250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:25.432019 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:25.432340 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:25.432837 (+ 497us) server_negotiation.cc:299] Negotiation successful
0927 23:02:25.433278 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":431,"threads_started":1}
I20260927 23:02:25.435532 21503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:25.540536 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 21379
2026-09-27T23:02:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65520 ms)
[----------] 30 tests from SecurityITest (323745 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:02:25 dist-test-slave-5pd3 krb5kdc[21565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:02:25 dist-test-slave-5pd3 krb5kdc[21565](info): set up 2 sockets
Sep 27 23:02:25 dist-test-slave-5pd3 krb5kdc[21565](info): commencing operation
krb5kdc: starting...
W20260927 23:02:26.724711 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:02:26 dist-test-slave-5pd3 krb5kdc[21565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:02:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:02:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:02:27.029074 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36659
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:27.398099 21578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:27.398445 21578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:27.398548 21578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:27.406154 21578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:27.406296 21578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:27.406349 21578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:27.406400 21578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:27.406452 21578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:27.417068 21578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:27.418596 21578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:27.420047 21578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:27.434829 21586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:27.434870 21583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:27.435288 21578 server_base.cc:1034] running on GCE node
W20260927 23:02:27.435026 21584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:27.436693 21578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:27.438256 21578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:27.439523 21578 hybrid_clock.cc:648] HybridClock initialized: now 1790550147439460 us; error 49 us; skew 500 ppm
I20260927 23:02:27.439997 21578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:27 dist-test-slave-5pd3 krb5kdc[21565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550147, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:27.443632 21578 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:02:27.445441 21578 webserver.cc:460] Webserver started at http://127.4.74.190:45797/ using document root <none> and password file <none>
I20260927 23:02:27.446183 21578 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:27.446363 21578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:27.446694 21578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:27.449972 21578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "4aaa70416c7949f993bb040c26afd006"
format_stamp: "Formatted at 2026-09-27 23:02:27 on dist-test-slave-5pd3"
server_key: "1085ecd85d720ed17716dd506fee90c3"
server_key_iv: "de356bcb3beb72e11bf6830706def860"
server_key_version: "encryptionkey@0"
I20260927 23:02:27.450762 21578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "4aaa70416c7949f993bb040c26afd006"
format_stamp: "Formatted at 2026-09-27 23:02:27 on dist-test-slave-5pd3"
server_key: "1085ecd85d720ed17716dd506fee90c3"
server_key_iv: "de356bcb3beb72e11bf6830706def860"
server_key_version: "encryptionkey@0"
I20260927 23:02:27.456761 21578 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260927 23:02:27.461349 21593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:27.463176 21578 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260927 23:02:27.463451 21578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "4aaa70416c7949f993bb040c26afd006"
format_stamp: "Formatted at 2026-09-27 23:02:27 on dist-test-slave-5pd3"
server_key: "1085ecd85d720ed17716dd506fee90c3"
server_key_iv: "de356bcb3beb72e11bf6830706def860"
server_key_version: "encryptionkey@0"
I20260927 23:02:27.463820 21578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:27.498945 21578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:27.504273 21578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:27.504614 21578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:27.529166 21578 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:36659
I20260927 23:02:27.529197 21644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:36659 every 8 connection(s)
I20260927 23:02:27.530962 21578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:27.531672 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 21578
I20260927 23:02:27.531983 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:02:27.532894 4394 external_mini_cluster.cc:1428] Setting key 3aafc6f2775824fb5d3cf77a45c4bae9
I20260927 23:02:27.536360 21645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 23:02:27 dist-test-slave-5pd3 krb5kdc[21565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:27.552325 21645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006: Bootstrap starting.
I20260927 23:02:27.555994 21645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:27.557199 21645 log.cc:824] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:27.561151 21645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006: No bootstrap required, opened a new log
I20260927 23:02:27.562294 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:27.535481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.4.74.190:36659)
0927 23:02:27.536581 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:27.536600 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:27.536666 (+ 66us) server_negotiation.cc:407] Connection header received
0927 23:02:27.538630 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:27.538660 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:27.539240 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:27.539830 (+ 590us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:27.541844 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:27.543609 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:27.546144 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:27.547093 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:27.547295 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:27.553132 (+ 5837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:27.553143 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:27.553166 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:27.553230 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:27.557029 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:27.558104 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:27.558114 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:27.558124 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:27.558406 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:27.558871 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:27.558878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:27.558881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:27.559634 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:27.559901 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:27.560444 (+ 543us) server_negotiation.cc:299] Negotiation successful
0927 23:02:27.560954 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":433,"threads_started":1}
I20260927 23:02:27.570713 21645 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aaa70416c7949f993bb040c26afd006" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36659 } }
I20260927 23:02:27.571156 21645 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:27.571271 21645 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aaa70416c7949f993bb040c26afd006, State: Initialized, Role: FOLLOWER
I20260927 23:02:27.571928 21645 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aaa70416c7949f993bb040c26afd006" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36659 } }
I20260927 23:02:27.572177 21645 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:27.572348 21645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:27.572527 21645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:27.575155 21645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aaa70416c7949f993bb040c26afd006" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36659 } }
I20260927 23:02:27.575878 21645 leader_election.cc:302] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4aaa70416c7949f993bb040c26afd006; no voters:
I20260927 23:02:27.577160 21645 leader_election.cc:288] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:02:27.577394 21650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:27.578291 21650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [term 1 LEADER]: Becoming Leader. State: Replica: 4aaa70416c7949f993bb040c26afd006, State: Running, Role: LEADER
I20260927 23:02:27.578969 21650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aaa70416c7949f993bb040c26afd006" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36659 } }
I20260927 23:02:27.579512 21645 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:27.582479 21652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4aaa70416c7949f993bb040c26afd006. Latest consensus state: current_term: 1 leader_uuid: "4aaa70416c7949f993bb040c26afd006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aaa70416c7949f993bb040c26afd006" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36659 } } }
I20260927 23:02:27.583019 21652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:27.582911 21651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4aaa70416c7949f993bb040c26afd006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aaa70416c7949f993bb040c26afd006" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36659 } } }
I20260927 23:02:27.583335 21651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:27.588446 21659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:27.596637 21659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:27.608767 21659 catalog_manager.cc:1347] Generated new cluster ID: 4913d72668374df39ef4f30fc2c7b5ee
I20260927 23:02:27.608939 21659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:27.621963 21659 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:02:27.623445 21659 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:27.637514 21659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006: Generated new TSK 0
I20260927 23:02:27.639261 21659 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 23:02:27.709041 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36659
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:28.081774 21671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:28.082094 21671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:28.082253 21671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:28.090173 21671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:28.090293 21671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:28.090466 21671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:02:28.103308 21671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36659
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:28.105027 21671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:28.106768 21671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:28.123855 21676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:28.123932 21677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:28.123951 21679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:28.124949 21671 server_base.cc:1034] running on GCE node
I20260927 23:02:28.125808 21671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:28.127198 21671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:28.128492 21671 hybrid_clock.cc:648] HybridClock initialized: now 1790550148128427 us; error 60 us; skew 500 ppm
I20260927 23:02:28.129069 21671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:28 dist-test-slave-5pd3 krb5kdc[21565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550148, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:28.133319 21671 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:02:28.135221 21671 webserver.cc:460] Webserver started at http://127.4.74.129:37527/ using document root <none> and password file <none>
I20260927 23:02:28.136241 21671 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:28.136423 21671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:28.136771 21671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:28.140693 21671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "e722d5123e414efcb9ec6ff439cc7fcc"
format_stamp: "Formatted at 2026-09-27 23:02:28 on dist-test-slave-5pd3"
server_key: "9b90322149ed4c6fe0cf0ab414aa53e4"
server_key_iv: "143f8cbfeac6df13307446cea4e0a138"
server_key_version: "encryptionkey@0"
I20260927 23:02:28.141989 21671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "e722d5123e414efcb9ec6ff439cc7fcc"
format_stamp: "Formatted at 2026-09-27 23:02:28 on dist-test-slave-5pd3"
server_key: "9b90322149ed4c6fe0cf0ab414aa53e4"
server_key_iv: "143f8cbfeac6df13307446cea4e0a138"
server_key_version: "encryptionkey@0"
I20260927 23:02:28.149060 21671 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260927 23:02:28.154356 21686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:28.156558 21671 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 23:02:28.156930 21671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "e722d5123e414efcb9ec6ff439cc7fcc"
format_stamp: "Formatted at 2026-09-27 23:02:28 on dist-test-slave-5pd3"
server_key: "9b90322149ed4c6fe0cf0ab414aa53e4"
server_key_iv: "143f8cbfeac6df13307446cea4e0a138"
server_key_version: "encryptionkey@0"
I20260927 23:02:28.157232 21671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:28.182617 21671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:28.188930 21671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:28.189327 21671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:28.190857 21671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:28.192955 21671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:28.193068 21671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:28.193185 21671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:28.193234 21671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:28.237707 21671 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:34345
I20260927 23:02:28.237936 21798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:34345 every 8 connection(s)
I20260927 23:02:28.239888 21671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:02:28.244240 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 21671
I20260927 23:02:28.244505 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 23:02:28.245266 4394 external_mini_cluster.cc:1428] Setting key b1ba180b63c76645cae5209e3e8079ce
Sep 27 23:02:28 dist-test-slave-5pd3 krb5kdc[21565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550148, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:28.268131 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:28.243790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:55173 (local address 127.4.74.190:36659)
0927 23:02:28.244457 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:28.244465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:28.245632 (+ 1167us) server_negotiation.cc:407] Connection header received
0927 23:02:28.248409 (+ 2777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:28.248413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:28.248507 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:28.248625 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:28.251168 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:28.252143 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:28.254916 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:28.255471 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:28.255803 (+ 332us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:28.260342 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:28.260353 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:28.260362 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:28.260428 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:28.264565 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:28.265514 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:28.265523 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:28.265527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:28.265670 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:28.266143 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:28.266150 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:28.266159 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:28.266462 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:28.266621 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:28.267400 (+ 779us) server_negotiation.cc:299] Negotiation successful
0927 23:02:28.267690 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":335,"threads_started":1}
I20260927 23:02:28.269276 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:28.244345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36659 (local address 127.4.74.129:55173)
0927 23:02:28.245406 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:28.245459 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:28.248021 (+ 2562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:28.248987 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:28.249000 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:28.249951 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:28.250907 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:28.250938 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:28.252351 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:28.252359 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:28.254630 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:28.254653 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:28.255946 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:28.255956 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:28.256341 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:28.257346 (+ 1005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:02:28.257404 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:28.259993 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:28.264832 (+ 4839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:28.264844 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:28.264867 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:28.265375 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:28.265801 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:28.265807 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:28.265815 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:28.266008 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:28.266602 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:28.266610 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:28.266992 (+ 382us) client_negotiation.cc:769] Sending connection context
0927 23:02:28.267296 (+ 304us) client_negotiation.cc:240] Negotiation successful
0927 23:02:28.267911 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":487,"threads_started":1}
I20260927 23:02:28.271492 21799 heartbeater.cc:347] Connected to a master server at 127.4.74.190:36659
I20260927 23:02:28.272002 21799 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:28.272914 21799 heartbeater.cc:511] Master 127.4.74.190:36659 requested a full tablet report, sending...
I20260927 23:02:28.275636 21610 ts_manager.cc:194] Registered new tserver with Master: e722d5123e414efcb9ec6ff439cc7fcc (127.4.74.129:34345)
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 23:02:28.277323 21610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:55173
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 23:02:28.316897 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36659
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:28.690019 21805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:28.690347 21805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:28.690493 21805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:28.698378 21805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:28.698531 21805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:28.698765 21805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:02:28.709069 21805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36659
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:28.710561 21805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:28.712036 21805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:28.727100 21810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:28.727165 21813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:28.727047 21811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:28.728534 21805 server_base.cc:1034] running on GCE node
I20260927 23:02:28.729312 21805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:28.730599 21805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:28.731938 21805 hybrid_clock.cc:648] HybridClock initialized: now 1790550148731866 us; error 56 us; skew 500 ppm
I20260927 23:02:28.732414 21805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:28 dist-test-slave-5pd3 krb5kdc[21565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550148, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:28.736977 21805 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:02:28.739092 21805 webserver.cc:460] Webserver started at http://127.4.74.130:44761/ using document root <none> and password file <none>
I20260927 23:02:28.740234 21805 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:28.740420 21805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:28.740839 21805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:28.744266 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "daf8c551e4b04fd1b9dac35327695353"
format_stamp: "Formatted at 2026-09-27 23:02:28 on dist-test-slave-5pd3"
server_key: "1fe992fc5a2fc528e03763041c5f51ab"
server_key_iv: "41dd82c95cbfb59fdf1e941b27db8bcf"
server_key_version: "encryptionkey@0"
I20260927 23:02:28.745515 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "daf8c551e4b04fd1b9dac35327695353"
format_stamp: "Formatted at 2026-09-27 23:02:28 on dist-test-slave-5pd3"
server_key: "1fe992fc5a2fc528e03763041c5f51ab"
server_key_iv: "41dd82c95cbfb59fdf1e941b27db8bcf"
server_key_version: "encryptionkey@0"
I20260927 23:02:28.752090 21805 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 23:02:28.756471 21820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:28.758412 21805 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:02:28.758631 21805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "daf8c551e4b04fd1b9dac35327695353"
format_stamp: "Formatted at 2026-09-27 23:02:28 on dist-test-slave-5pd3"
server_key: "1fe992fc5a2fc528e03763041c5f51ab"
server_key_iv: "41dd82c95cbfb59fdf1e941b27db8bcf"
server_key_version: "encryptionkey@0"
I20260927 23:02:28.758909 21805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:28.790099 21805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:28.795061 21805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:28.795415 21805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:28.796715 21805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:28.798280 21805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:28.798393 21805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:28.798547 21805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:28.798619 21805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:28.839821 21805 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:43573
I20260927 23:02:28.839905 21932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:43573 every 8 connection(s)
I20260927 23:02:28.841799 21805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:02:28.849489 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 21805
I20260927 23:02:28.849782 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 23:02:28.850538 4394 external_mini_cluster.cc:1428] Setting key 35c3b8d67005ef02ca1d492e36757b81
Sep 27 23:02:28 dist-test-slave-5pd3 krb5kdc[21565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550148, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:28.871308 21935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:28.845351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:41887 (local address 127.4.74.190:36659)
0927 23:02:28.845970 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:28.845977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:28.847264 (+ 1287us) server_negotiation.cc:407] Connection header received
0927 23:02:28.850127 (+ 2863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:28.850132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:28.850229 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:28.850378 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:28.853121 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:28.854000 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:28.857305 (+ 3305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:28.857804 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:28.857945 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:28.863832 (+ 5887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:28.863840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:28.863843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:28.863888 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:28.866931 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:28.868054 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:28.868058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:28.868060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:28.868202 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:28.868689 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:28.868703 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:28.868706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:28.868998 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:28.869194 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:28.870785 (+ 1591us) server_negotiation.cc:299] Negotiation successful
0927 23:02:28.871009 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":301,"threads_started":1}
I20260927 23:02:28.873762 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:28.845907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36659 (local address 127.4.74.130:41887)
0927 23:02:28.846953 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:28.847054 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:28.849600 (+ 2546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:28.850806 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:28.850823 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:28.851901 (+ 1078us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:28.852838 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:28.852890 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:28.854218 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:28.854226 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:28.857010 (+ 2784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:28.857044 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:28.858147 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:28.858157 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:28.858562 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:28.859633 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:02:28.859676 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:28.863377 (+ 3701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:28.867156 (+ 3779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:28.867167 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:28.867190 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:28.867835 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:28.868334 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:28.868339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:28.868342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:28.868543 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:28.869251 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:28.869266 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:28.870146 (+ 880us) client_negotiation.cc:769] Sending connection context
0927 23:02:28.870643 (+ 497us) client_negotiation.cc:240] Negotiation successful
0927 23:02:28.871490 (+ 847us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":386,"threads_started":1}
I20260927 23:02:28.875869 21933 heartbeater.cc:347] Connected to a master server at 127.4.74.190:36659
I20260927 23:02:28.876375 21933 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:28.877580 21933 heartbeater.cc:511] Master 127.4.74.190:36659 requested a full tablet report, sending...
I20260927 23:02:28.879858 21610 ts_manager.cc:194] Registered new tserver with Master: daf8c551e4b04fd1b9dac35327695353 (127.4.74.130:43573)
I20260927 23:02:28.880903 21610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:41887
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 23:02:28.929167 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36659
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:02:29.281606 21799 heartbeater.cc:503] Master 127.4.74.190:36659 was elected leader, sending a full tablet report...
W20260927 23:02:29.306108 21939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:29.306512 21939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:29.306690 21939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:29.314337 21939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:29.314543 21939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:29.314749 21939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:02:29.325824 21939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36659
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:29.327311 21939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:29.328729 21939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:29.344941 21944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:29.345265 21947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:29.345464 21945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:29.346374 21939 server_base.cc:1034] running on GCE node
I20260927 23:02:29.347266 21939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:29.348574 21939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:29.350046 21939 hybrid_clock.cc:648] HybridClock initialized: now 1790550149349978 us; error 58 us; skew 500 ppm
I20260927 23:02:29.350467 21939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:29 dist-test-slave-5pd3 krb5kdc[21565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550149, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:29.354681 21939 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:02:29.356781 21939 webserver.cc:460] Webserver started at http://127.4.74.131:41651/ using document root <none> and password file <none>
I20260927 23:02:29.357622 21939 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:29.357756 21939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:29.358057 21939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:29.361296 21939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "d59440b784c94b86b59fbf36c2081e3f"
format_stamp: "Formatted at 2026-09-27 23:02:29 on dist-test-slave-5pd3"
server_key: "ff1d301299fd8cbd132ddc372e787b97"
server_key_iv: "8cfdcac3e632a4c49a88180534875df4"
server_key_version: "encryptionkey@0"
I20260927 23:02:29.362102 21939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "d59440b784c94b86b59fbf36c2081e3f"
format_stamp: "Formatted at 2026-09-27 23:02:29 on dist-test-slave-5pd3"
server_key: "ff1d301299fd8cbd132ddc372e787b97"
server_key_iv: "8cfdcac3e632a4c49a88180534875df4"
server_key_version: "encryptionkey@0"
I20260927 23:02:29.368882 21939 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260927 23:02:29.373306 21954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:29.375267 21939 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:02:29.375517 21939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "d59440b784c94b86b59fbf36c2081e3f"
format_stamp: "Formatted at 2026-09-27 23:02:29 on dist-test-slave-5pd3"
server_key: "ff1d301299fd8cbd132ddc372e787b97"
server_key_iv: "8cfdcac3e632a4c49a88180534875df4"
server_key_version: "encryptionkey@0"
I20260927 23:02:29.375793 21939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:29.391239 21939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:29.396251 21939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:29.396601 21939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:29.397786 21939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:29.399384 21939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:29.399510 21939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:29.399693 21939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:29.399781 21939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:29.439175 21939 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:35197
I20260927 23:02:29.439320 22066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:35197 every 8 connection(s)
I20260927 23:02:29.441057 21939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:02:29.442713 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 21939
I20260927 23:02:29.442922 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 23:02:29.443421 4394 external_mini_cluster.cc:1428] Setting key d5371a38b3d7a6973907f61d045251bd
Sep 27 23:02:29 dist-test-slave-5pd3 krb5kdc[21565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550149, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:29.465689 22069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.444489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:46717 (local address 127.4.74.190:36659)
0927 23:02:29.445159 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:29.445167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:29.446662 (+ 1495us) server_negotiation.cc:407] Connection header received
0927 23:02:29.448906 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:29.448911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:29.449008 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:29.449140 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:29.450880 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.451856 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.453490 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.453888 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.454026 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.458226 (+ 4200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:29.458232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:29.458235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:29.458280 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:29.462115 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:29.463036 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:29.463040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:29.463042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:29.463163 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:29.463693 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:29.463697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:29.463700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:29.464069 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:29.464237 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:29.465047 (+ 810us) server_negotiation.cc:299] Negotiation successful
0927 23:02:29.465298 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":365,"threads_started":1}
I20260927 23:02:29.466686 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.445392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36659 (local address 127.4.74.131:46717)
0927 23:02:29.446489 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:29.446543 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:29.448582 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:29.449414 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:29.449424 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:29.450061 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:29.450691 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.450709 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.452026 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.452030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.453321 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.453333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.454128 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.454135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.454378 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.455179 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:02:29.455212 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:29.457936 (+ 2724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:29.462314 (+ 4378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:29.462324 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:29.462342 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:29.462865 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:29.463277 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:29.463284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:29.463288 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:29.463493 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:29.464210 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:29.464219 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:29.464600 (+ 381us) client_negotiation.cc:769] Sending connection context
0927 23:02:29.464935 (+ 335us) client_negotiation.cc:240] Negotiation successful
0927 23:02:29.465510 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":450,"threads_started":1}
I20260927 23:02:29.468900 22067 heartbeater.cc:347] Connected to a master server at 127.4.74.190:36659
I20260927 23:02:29.469515 22067 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:29.470618 22067 heartbeater.cc:511] Master 127.4.74.190:36659 requested a full tablet report, sending...
I20260927 23:02:29.472770 21610 ts_manager.cc:194] Registered new tserver with Master: d59440b784c94b86b59fbf36c2081e3f (127.4.74.131:35197)
I20260927 23:02:29.474128 21610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:46717
I20260927 23:02:29.475330 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:02:29.496454 22069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.481211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44590 (local address 127.4.74.190:36659)
0927 23:02:29.481421 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:29.481426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:29.482012 (+ 586us) server_negotiation.cc:407] Connection header received
0927 23:02:29.482143 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:29.482148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:29.482229 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:29.482393 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:29.484044 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.484892 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.486505 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.487151 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.487292 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.489997 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:29.490003 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:29.490006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:29.490046 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:29.493083 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:29.493981 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:29.493986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:29.493988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:29.494132 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:29.494574 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:29.494578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:29.494581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:29.495042 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:29.495289 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:29.495842 (+ 553us) server_negotiation.cc:299] Negotiation successful
0927 23:02:29.496165 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260927 23:02:29.501873 21609 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:02:29.509586 21609 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:02:29.555992 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.525695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:35197 (local address 127.0.0.1:57526)
0927 23:02:29.542276 (+ 16581us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:29.542303 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:29.542431 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:29.543310 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:29.543315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:29.543341 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:29.543945 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.543969 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.546028 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.546039 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.551955 (+ 5916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.551987 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.554716 (+ 2729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.554723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.555001 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.555060 (+ 59us) client_negotiation.cc:769] Sending connection context
0927 23:02:29.555216 (+ 156us) client_negotiation.cc:240] Negotiation successful
0927 23:02:29.555463 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16449,"mutex_wait_us":602,"thread_start_us":15794,"threads_started":1}
I20260927 23:02:29.557421 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.525407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:34345 (local address 127.0.0.1:38336)
0927 23:02:29.540959 (+ 15552us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:29.541034 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:29.541269 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:29.542293 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:29.542301 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:29.542340 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:29.542844 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.542858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.545847 (+ 2989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.545855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.549117 (+ 3262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.549151 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.556291 (+ 7140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.556300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.556566 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.556617 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 23:02:29.556743 (+ 126us) client_negotiation.cc:240] Negotiation successful
0927 23:02:29.556915 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15147,"mutex_wait_us":1,"thread_start_us":20411,"threads_started":1}
I20260927 23:02:29.557440 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.525933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57526 (local address 127.4.74.131:35197)
0927 23:02:29.541619 (+ 15686us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:29.541628 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:29.542573 (+ 945us) server_negotiation.cc:407] Connection header received
0927 23:02:29.542685 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:29.542695 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:29.543021 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:29.543221 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:29.544214 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.545834 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.552267 (+ 6433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.554104 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.554253 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.554565 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:29.556798 (+ 2233us) server_negotiation.cc:299] Negotiation successful
0927 23:02:29.557063 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15477,"thread_start_us":15683,"threads_started":1}
I20260927 23:02:29.559758 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.526106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38336 (local address 127.4.74.129:34345)
0927 23:02:29.541394 (+ 15288us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:29.541406 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:29.541453 (+ 47us) server_negotiation.cc:407] Connection header received
0927 23:02:29.541599 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:29.541610 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:29.541955 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:29.542282 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:29.543661 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.545629 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.552749 (+ 7120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.555012 (+ 2263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.555663 (+ 651us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.556042 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:29.559136 (+ 3094us) server_negotiation.cc:299] Negotiation successful
0927 23:02:29.559372 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15048,"thread_start_us":16900,"threads_started":1}
I20260927 23:02:29.563108 22002 tablet_service.cc:1467] Processing CreateTablet for tablet a53a6bae36ee4f29a0acb37ecfd8a571 (DEFAULT_TABLE table=test-table [id=cfc42149fe5545128a57409e5b97c565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:29.564312 21734 tablet_service.cc:1467] Processing CreateTablet for tablet a53a6bae36ee4f29a0acb37ecfd8a571 (DEFAULT_TABLE table=test-table [id=cfc42149fe5545128a57409e5b97c565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:29.565213 22002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53a6bae36ee4f29a0acb37ecfd8a571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:29.565699 21734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53a6bae36ee4f29a0acb37ecfd8a571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:29.566376 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.549134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:43573 (local address 127.0.0.1:54210)
0927 23:02:29.556383 (+ 7249us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:29.556410 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:29.556579 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:29.557661 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:29.557666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:29.557702 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:29.557898 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.557915 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.560837 (+ 2922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.560843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.562854 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.562875 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.565495 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.565502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.565711 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.565749 (+ 38us) client_negotiation.cc:769] Sending connection context
0927 23:02:29.565859 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 23:02:29.566058 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7127,"thread_start_us":11038,"threads_started":1}
I20260927 23:02:29.567539 22084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.546743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.4.74.130:43573)
0927 23:02:29.555631 (+ 8888us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:29.555641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:29.556647 (+ 1006us) server_negotiation.cc:407] Connection header received
0927 23:02:29.556791 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:29.556800 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:29.557119 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:29.557345 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:29.559045 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.560651 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.563124 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.564940 (+ 1816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.565115 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.565352 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:29.566856 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0927 23:02:29.567079 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8679,"thread_start_us":8533,"threads_started":1}
I20260927 23:02:29.571979 21868 tablet_service.cc:1467] Processing CreateTablet for tablet a53a6bae36ee4f29a0acb37ecfd8a571 (DEFAULT_TABLE table=test-table [id=cfc42149fe5545128a57409e5b97c565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:29.573632 21868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53a6bae36ee4f29a0acb37ecfd8a571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:29.581573 22086 tablet_bootstrap.cc:492] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: Bootstrap starting.
I20260927 23:02:29.584612 22087 tablet_bootstrap.cc:492] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: Bootstrap starting.
I20260927 23:02:29.586800 22086 tablet_bootstrap.cc:654] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:29.588405 22086 log.cc:824] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:29.588701 22087 tablet_bootstrap.cc:654] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:29.589334 22088 tablet_bootstrap.cc:492] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: Bootstrap starting.
I20260927 23:02:29.590313 22087 log.cc:824] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:29.593044 22088 tablet_bootstrap.cc:654] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:29.594370 22088 log.cc:824] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:29.594398 22086 tablet_bootstrap.cc:492] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: No bootstrap required, opened a new log
I20260927 23:02:29.594398 22087 tablet_bootstrap.cc:492] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: No bootstrap required, opened a new log
I20260927 23:02:29.594759 22087 ts_tablet_manager.cc:1397] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260927 23:02:29.594766 22086 ts_tablet_manager.cc:1397] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260927 23:02:29.603451 22088 tablet_bootstrap.cc:492] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: No bootstrap required, opened a new log
I20260927 23:02:29.603855 22088 ts_tablet_manager.cc:1397] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.007s
I20260927 23:02:29.607062 22087 raft_consensus.cc:348] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34345 } } peers { permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 } } peers { permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 } }
I20260927 23:02:29.607062 22086 raft_consensus.cc:348] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34345 } } peers { permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 } } peers { permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 } }
I20260927 23:02:29.607710 22086 raft_consensus.cc:374] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:29.607710 22087 raft_consensus.cc:374] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:29.607829 22086 raft_consensus.cc:729] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d59440b784c94b86b59fbf36c2081e3f, State: Initialized, Role: FOLLOWER
I20260927 23:02:29.607829 22087 raft_consensus.cc:729] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e722d5123e414efcb9ec6ff439cc7fcc, State: Initialized, Role: FOLLOWER
I20260927 23:02:29.608682 22087 consensus_queue.cc:260] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34345 } } peers { permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 } } peers { permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 } }
I20260927 23:02:29.608676 22086 consensus_queue.cc:260] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34345 } } peers { permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 } } peers { permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 } }
I20260927 23:02:29.612234 22087 ts_tablet_manager.cc:1428] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: Time spent starting tablet: real 0.017s user 0.019s sys 0.000s
I20260927 23:02:29.612213 22086 ts_tablet_manager.cc:1428] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: Time spent starting tablet: real 0.017s user 0.018s sys 0.000s
I20260927 23:02:29.612913 22067 heartbeater.cc:503] Master 127.4.74.190:36659 was elected leader, sending a full tablet report...
I20260927 23:02:29.612797 22088 raft_consensus.cc:348] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34345 } } peers { permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 } } peers { permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 } }
I20260927 23:02:29.613590 22088 raft_consensus.cc:374] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:29.613718 22088 raft_consensus.cc:729] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daf8c551e4b04fd1b9dac35327695353, State: Initialized, Role: FOLLOWER
I20260927 23:02:29.614421 22088 consensus_queue.cc:260] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34345 } } peers { permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 } } peers { permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 } }
I20260927 23:02:29.615840 21933 heartbeater.cc:503] Master 127.4.74.190:36659 was elected leader, sending a full tablet report...
I20260927 23:02:29.616587 22088 ts_tablet_manager.cc:1428] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: Time spent starting tablet: real 0.013s user 0.010s sys 0.001s
I20260927 23:02:29.667359 22092 raft_consensus.cc:482] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:02:29.667764 22092 raft_consensus.cc:504] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34345 } } peers { permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 } } peers { permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 } }
I20260927 23:02:29.670225 22092 leader_election.cc:288] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d59440b784c94b86b59fbf36c2081e3f (127.4.74.131:35197), daf8c551e4b04fd1b9dac35327695353 (127.4.74.130:43573)
I20260927 23:02:29.680050 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.670959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:52005 (local address 127.4.74.131:35197)
0927 23:02:29.671180 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:29.671188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:29.672325 (+ 1137us) server_negotiation.cc:407] Connection header received
0927 23:02:29.672533 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:29.672540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:29.672647 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:29.672775 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:29.673867 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.675064 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.677946 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.678888 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.679101 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.679202 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:29.679573 (+ 371us) server_negotiation.cc:299] Negotiation successful
0927 23:02:29.679712 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260927 23:02:29.680253 22095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.670899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:35197 (local address 127.4.74.129:52005)
0927 23:02:29.672054 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:29.672120 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:29.672347 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:29.672852 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:29.672858 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:29.672921 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:29.673608 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.673631 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.675295 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.675301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.677728 (+ 2427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.677755 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.679129 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.679136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.679352 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.679385 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 23:02:29.679521 (+ 136us) client_negotiation.cc:240] Negotiation successful
0927 23:02:29.679705 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":86,"spinlock_wait_cycles":83072,"thread_start_us":627,"threads_started":1}
I20260927 23:02:29.680722 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.670899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:43573 (local address 127.4.74.129:43577)
0927 23:02:29.672130 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:29.672149 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:29.672347 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:29.672852 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:29.672858 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:29.672920 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:29.673636 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.673645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.675294 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.675301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.677734 (+ 2433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:29.677756 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.679395 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.679402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:29.679673 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.679702 (+ 29us) client_negotiation.cc:769] Sending connection context
0927 23:02:29.679827 (+ 125us) client_negotiation.cc:240] Negotiation successful
0927 23:02:29.680004 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"mutex_wait_us":108,"spinlock_wait_cycles":58752,"thread_start_us":623,"threads_started":1}
I20260927 23:02:29.680821 22084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.670913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:43577 (local address 127.4.74.130:43573)
0927 23:02:29.671181 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:29.671188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:29.672325 (+ 1137us) server_negotiation.cc:407] Connection header received
0927 23:02:29.672533 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:29.672540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:29.672638 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:29.672775 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:29.673870 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.675064 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.677998 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.679077 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.679306 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.679505 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:29.679859 (+ 354us) server_negotiation.cc:299] Negotiation successful
0927 23:02:29.680015 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 23:02:29.681753 22022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53a6bae36ee4f29a0acb37ecfd8a571" candidate_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d59440b784c94b86b59fbf36c2081e3f" is_pre_election: true
I20260927 23:02:29.682179 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53a6bae36ee4f29a0acb37ecfd8a571" candidate_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daf8c551e4b04fd1b9dac35327695353" is_pre_election: true
I20260927 23:02:29.682503 22022 raft_consensus.cc:2454] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e722d5123e414efcb9ec6ff439cc7fcc in term 0.
I20260927 23:02:29.682746 21888 raft_consensus.cc:2454] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e722d5123e414efcb9ec6ff439cc7fcc in term 0.
I20260927 23:02:29.683446 21690 leader_election.cc:302] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d59440b784c94b86b59fbf36c2081e3f, e722d5123e414efcb9ec6ff439cc7fcc; no voters:
I20260927 23:02:29.684053 22092 raft_consensus.cc:2781] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:02:29.684293 22092 raft_consensus.cc:482] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:02:29.684437 22092 raft_consensus.cc:3037] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:29.687317 22092 raft_consensus.cc:504] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34345 } } peers { permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 } } peers { permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 } }
I20260927 23:02:29.688228 22092 leader_election.cc:288] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [CANDIDATE]: Term 1 election: Requested vote from peers d59440b784c94b86b59fbf36c2081e3f (127.4.74.131:35197), daf8c551e4b04fd1b9dac35327695353 (127.4.74.130:43573)
I20260927 23:02:29.688858 22022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53a6bae36ee4f29a0acb37ecfd8a571" candidate_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d59440b784c94b86b59fbf36c2081e3f"
I20260927 23:02:29.688941 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53a6bae36ee4f29a0acb37ecfd8a571" candidate_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daf8c551e4b04fd1b9dac35327695353"
I20260927 23:02:29.689201 22022 raft_consensus.cc:3037] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:29.689227 21888 raft_consensus.cc:3037] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:29.691890 22022 raft_consensus.cc:2454] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e722d5123e414efcb9ec6ff439cc7fcc in term 1.
I20260927 23:02:29.692078 21888 raft_consensus.cc:2454] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e722d5123e414efcb9ec6ff439cc7fcc in term 1.
I20260927 23:02:29.692571 21690 leader_election.cc:302] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d59440b784c94b86b59fbf36c2081e3f, e722d5123e414efcb9ec6ff439cc7fcc; no voters:
W20260927 23:02:29.692890 22068 tablet.cc:2369] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:29.692950 22092 raft_consensus.cc:2781] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:29.693522 22092 raft_consensus.cc:686] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 1 LEADER]: Becoming Leader. State: Replica: e722d5123e414efcb9ec6ff439cc7fcc, State: Running, Role: LEADER
I20260927 23:02:29.694224 22092 consensus_queue.cc:237] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34345 } } peers { permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 } } peers { permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 } }
I20260927 23:02:29.700135 21610 catalog_manager.cc:5495] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc reported cstate change: term changed from 0 to 1, leader changed from <none> to e722d5123e414efcb9ec6ff439cc7fcc (127.4.74.129). New cstate: current_term: 1 leader_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e722d5123e414efcb9ec6ff439cc7fcc" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:02:29.738049 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.732071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38340 (local address 127.4.74.129:34345)
0927 23:02:29.732377 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:29.732387 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:29.732486 (+ 99us) server_negotiation.cc:407] Connection header received
0927 23:02:29.732759 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:29.732764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:29.732865 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:29.733044 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:02:29.733428 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.734157 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.735653 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.736086 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.736243 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.736818 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:02:29.737397 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:02:29.737596 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:29.737650 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 23:02:29.737738 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260927 23:02:29.743355 21800 tablet.cc:2369] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:29.748649 22022 raft_consensus.cc:1261] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f [term 1 FOLLOWER]: Refusing update from remote peer e722d5123e414efcb9ec6ff439cc7fcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:29.748649 21888 raft_consensus.cc:1261] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353 [term 1 FOLLOWER]: Refusing update from remote peer e722d5123e414efcb9ec6ff439cc7fcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:29.750164 22097 consensus_queue.cc:1035] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d59440b784c94b86b59fbf36c2081e3f" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:29.750648 22092 consensus_queue.cc:1035] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "daf8c551e4b04fd1b9dac35327695353" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:29.765136 22100 mvcc.cc:205] Tried to move back new op lower bound from 7334093413356236800 to 7334093413152190464. Current Snapshot: MvccSnapshot[applied={T|T < 7334093413356236800}]
I20260927 23:02:29.785542 21610 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:29.785890 21610 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:29.790772 21610 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4aaa70416c7949f993bb040c26afd006: Sending DeleteTablet for 3 replicas of tablet a53a6bae36ee4f29a0acb37ecfd8a571
I20260927 23:02:29.792460 21868 tablet_service.cc:1514] Processing DeleteTablet for tablet a53a6bae36ee4f29a0acb37ecfd8a571 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:29 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:54210
I20260927 23:02:29.792441 22002 tablet_service.cc:1514] Processing DeleteTablet for tablet a53a6bae36ee4f29a0acb37ecfd8a571 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:29 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:57526
I20260927 23:02:29.793576 21734 tablet_service.cc:1514] Processing DeleteTablet for tablet a53a6bae36ee4f29a0acb37ecfd8a571 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:29 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:38336
I20260927 23:02:29.794183 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 21671
I20260927 23:02:29.802009 22112 tablet_replica.cc:330] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: stopping tablet replica
I20260927 23:02:29.802232 22110 tablet_replica.cc:330] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: stopping tablet replica
I20260927 23:02:29.802529 22112 raft_consensus.cc:2229] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:02:29.802898 22110 raft_consensus.cc:2229] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:29.803298 22112 raft_consensus.cc:2258] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:29.803735 22110 raft_consensus.cc:2258] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:29.805436 22111 tablet_replica.cc:330] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: stopping tablet replica
I20260927 23:02:29.805768 22112 ts_tablet_manager.cc:1905] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:29.806221 22111 raft_consensus.cc:2229] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:29.807055 22111 raft_consensus.cc:2258] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:29.810211 22111 ts_tablet_manager.cc:1905] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:29.811470 22110 ts_tablet_manager.cc:1905] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:29.813814 22112 ts_tablet_manager.cc:1918] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:29.814015 22112 log.cc:1196] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/a53a6bae36ee4f29a0acb37ecfd8a571
I20260927 23:02:29.814569 22112 ts_tablet_manager.cc:1939] T a53a6bae36ee4f29a0acb37ecfd8a571 P e722d5123e414efcb9ec6ff439cc7fcc: Deleting consensus metadata
Sep 27 23:02:29 dist-test-slave-5pd3 krb5kdc[21565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 23:02:29.817504 21595 catalog_manager.cc:4841] TS e722d5123e414efcb9ec6ff439cc7fcc (127.4.74.129:34345): tablet a53a6bae36ee4f29a0acb37ecfd8a571 (table test-table [id=cfc42149fe5545128a57409e5b97c565]) successfully deleted
I20260927 23:02:29.820838 22111 ts_tablet_manager.cc:1918] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:29.821238 22111 log.cc:1196] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/a53a6bae36ee4f29a0acb37ecfd8a571
I20260927 23:02:29.821822 22111 ts_tablet_manager.cc:1939] T a53a6bae36ee4f29a0acb37ecfd8a571 P d59440b784c94b86b59fbf36c2081e3f: Deleting consensus metadata
I20260927 23:02:29.824129 21597 catalog_manager.cc:4841] TS d59440b784c94b86b59fbf36c2081e3f (127.4.74.131:35197): tablet a53a6bae36ee4f29a0acb37ecfd8a571 (table test-table [id=cfc42149fe5545128a57409e5b97c565]) successfully deleted
I20260927 23:02:29.826617 22110 ts_tablet_manager.cc:1918] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:29.826848 22110 log.cc:1196] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/a53a6bae36ee4f29a0acb37ecfd8a571
I20260927 23:02:29.827288 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.795719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38348 (local address 127.4.74.129:34345)
0927 23:02:29.795915 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:29.795922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:29.797946 (+ 2024us) server_negotiation.cc:407] Connection header received
0927 23:02:29.798391 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:29.798401 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:29.798544 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:29.798661 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:29.804326 (+ 5665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.805731 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.809975 (+ 4244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.810767 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.810905 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.818323 (+ 7418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:29.818339 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:29.818360 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:29.818418 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:29.822920 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:29.824369 (+ 1449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:29.824385 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:29.824388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:29.824682 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:29.825149 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:29.825154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:29.825163 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:29.825741 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:29.826172 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:29.826491 (+ 319us) server_negotiation.cc:299] Negotiation successful
0927 23:02:29.826807 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":61}
I20260927 23:02:29.827450 22110 ts_tablet_manager.cc:1939] T a53a6bae36ee4f29a0acb37ecfd8a571 P daf8c551e4b04fd1b9dac35327695353: Deleting consensus metadata
I20260927 23:02:29.828408 21794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:29.829733 21595 catalog_manager.cc:4841] TS daf8c551e4b04fd1b9dac35327695353 (127.4.74.130:43573): tablet a53a6bae36ee4f29a0acb37ecfd8a571 (table test-table [id=cfc42149fe5545128a57409e5b97c565]) successfully deleted
I20260927 23:02:29.963634 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 21671
I20260927 23:02:29.986510 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 21805
Sep 27 23:02:29 dist-test-slave-5pd3 krb5kdc[21565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 23:02:30.006554 22084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:29.988011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54214 (local address 127.4.74.130:43573)
0927 23:02:29.988254 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:29.988258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:29.988283 (+ 25us) server_negotiation.cc:407] Connection header received
0927 23:02:29.988442 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:29.988447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:29.988528 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:29.988639 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:29.990153 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.990993 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.992672 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:29.993191 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:29.993317 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:29.998521 (+ 5204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:29.998544 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:29.998565 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:29.998629 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:30.002231 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:30.003294 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:30.003302 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:30.003305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:30.003452 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:30.004284 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:30.004291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:30.004299 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:30.004939 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:30.005386 (+ 447us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:30.005782 (+ 396us) server_negotiation.cc:299] Negotiation successful
0927 23:02:30.006071 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 23:02:30.007431 21928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:30.107811 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 21805
I20260927 23:02:30.126763 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 21939
Sep 27 23:02:30 dist-test-slave-5pd3 krb5kdc[21565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 23:02:30.144606 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:30.127826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57530 (local address 127.4.74.131:35197)
0927 23:02:30.128042 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:30.128046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:30.128070 (+ 24us) server_negotiation.cc:407] Connection header received
0927 23:02:30.128371 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:30.128377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:30.128472 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:30.128607 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:30.130167 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:30.130908 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:30.132428 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:30.132890 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:30.133006 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:30.138189 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:30.138212 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:30.138231 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:30.138278 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:30.141432 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:30.142271 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:30.142279 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:30.142282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:30.142429 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:30.142802 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:30.142805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:30.142807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:30.143169 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:30.143394 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:30.144072 (+ 678us) server_negotiation.cc:299] Negotiation successful
0927 23:02:30.144307 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 23:02:30.145296 22062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:30.248898 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 21939
I20260927 23:02:30.268285 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 21578
I20260927 23:02:30.269740 21640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:30.405582 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 21578
2026-09-27T23:02:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4876 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-27T23:02:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:02:30Z Disabled control of system clock
I20260927 23:02:30.506954 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:35175
--webserver_interface=127.4.74.190
--webserver_port=0
--builtin_ntp_servers=127.4.74.148:36621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:35175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 23:02:30.871795 22121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:30.872167 22121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:30.872255 22121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:30.880541 22121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:30.880686 22121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:30.880739 22121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:30.880811 22121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:30.880861 22121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:30.890769 22121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:35175
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:35175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:30.892241 22121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:30.894062 22121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:30.908442 22127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:30.908922 22126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:30.909237 22129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:30.910225 22121 server_base.cc:1034] running on GCE node
I20260927 23:02:30.911144 22121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:30.912814 22121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:30.914096 22121 hybrid_clock.cc:648] HybridClock initialized: now 1790550150914040 us; error 54 us; skew 500 ppm
I20260927 23:02:30.914539 22121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:30.917512 22121 webserver.cc:460] Webserver started at http://127.4.74.190:33745/ using document root <none> and password file <none>
I20260927 23:02:30.918351 22121 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:30.918493 22121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:30.918869 22121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:30.922514 22121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "5504470534f94108a83b205752ddcc63"
format_stamp: "Formatted at 2026-09-27 23:02:30 on dist-test-slave-5pd3"
server_key: "54647145ee6fb51a7040e88580c08f7f"
server_key_iv: "f9de72cb4af630de4fbe377cae60f4bd"
server_key_version: "encryptionkey@0"
I20260927 23:02:30.923506 22121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "5504470534f94108a83b205752ddcc63"
format_stamp: "Formatted at 2026-09-27 23:02:30 on dist-test-slave-5pd3"
server_key: "54647145ee6fb51a7040e88580c08f7f"
server_key_iv: "f9de72cb4af630de4fbe377cae60f4bd"
server_key_version: "encryptionkey@0"
I20260927 23:02:30.930289 22121 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:02:30.934825 22135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:30.938557 22121 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260927 23:02:30.938865 22121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "5504470534f94108a83b205752ddcc63"
format_stamp: "Formatted at 2026-09-27 23:02:30 on dist-test-slave-5pd3"
server_key: "54647145ee6fb51a7040e88580c08f7f"
server_key_iv: "f9de72cb4af630de4fbe377cae60f4bd"
server_key_version: "encryptionkey@0"
I20260927 23:02:30.939200 22121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:30.971494 22121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:30.972849 22121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:30.973249 22121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:30.995995 22121 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:35175
I20260927 23:02:30.996120 22186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:35175 every 8 connection(s)
I20260927 23:02:30.997565 22121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:30.999877 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 22121
I20260927 23:02:31.000094 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:02:31.000612 4394 external_mini_cluster.cc:1428] Setting key 7e4e5b6fc4459f305a6ac2afaaeaa555
I20260927 23:02:31.003814 22187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:31.018641 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:31.003039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.4.74.190:35175)
0927 23:02:31.004455 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:31.004473 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:31.004525 (+ 52us) server_negotiation.cc:407] Connection header received
0927 23:02:31.006440 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:31.006470 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:31.007068 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:31.007518 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:31.008466 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:31.010065 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:31.012631 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:31.013496 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:31.013684 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:31.015639 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:31.015653 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:31.015681 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:31.016359 (+ 678us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:31.016460 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:31.016484 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:31.016611 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:31.017053 (+ 442us) server_negotiation.cc:299] Negotiation successful
0927 23:02:31.017470 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":518,"threads_started":1}
I20260927 23:02:31.021636 22187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63: Bootstrap starting.
I20260927 23:02:31.025876 22187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:31.027004 22187 log.cc:824] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:31.030819 22187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63: No bootstrap required, opened a new log
I20260927 23:02:31.039201 22187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5504470534f94108a83b205752ddcc63" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35175 } }
I20260927 23:02:31.039716 22187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:31.039803 22187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5504470534f94108a83b205752ddcc63, State: Initialized, Role: FOLLOWER
I20260927 23:02:31.040455 22187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5504470534f94108a83b205752ddcc63" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35175 } }
I20260927 23:02:31.040784 22187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:31.041003 22187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:31.041242 22187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:31.043892 22187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5504470534f94108a83b205752ddcc63" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35175 } }
I20260927 23:02:31.044410 22187 leader_election.cc:302] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5504470534f94108a83b205752ddcc63; no voters:
I20260927 23:02:31.045089 22187 leader_election.cc:288] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:02:31.045395 22192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:31.046402 22192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [term 1 LEADER]: Becoming Leader. State: Replica: 5504470534f94108a83b205752ddcc63, State: Running, Role: LEADER
I20260927 23:02:31.047138 22192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5504470534f94108a83b205752ddcc63" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35175 } }
I20260927 23:02:31.047574 22187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:31.051308 22193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5504470534f94108a83b205752ddcc63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5504470534f94108a83b205752ddcc63" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35175 } } }
I20260927 23:02:31.051908 22193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:31.051194 22194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5504470534f94108a83b205752ddcc63. Latest consensus state: current_term: 1 leader_uuid: "5504470534f94108a83b205752ddcc63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5504470534f94108a83b205752ddcc63" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 35175 } } }
I20260927 23:02:31.052434 22194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63 [sys.catalog]: This master's current role is: LEADER
W20260927 23:02:31.060614 22208 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:02:31.060849 22208 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:02:31.061530 22201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:31.063830 22201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:31.075513 22201 catalog_manager.cc:1347] Generated new cluster ID: a9a51e737d314f94a94d70c41ecf8439
I20260927 23:02:31.075794 22201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:31.095330 22201 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:02:31.097004 22201 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:31.107098 22201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63: Generated new TSK 0
I20260927 23:02:31.108446 22201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:02:31.125363 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35175
--builtin_ntp_servers=127.4.74.148:36621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 23:02:31.548039 22211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:31.548449 22211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:31.548636 22211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:31.557979 22211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:31.558121 22211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:31.558320 22211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:02:31.569162 22211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:31.570676 22211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:31.572769 22211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:31.588058 22216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:31.588043 22217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:31.588126 22219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:31.589588 22211 server_base.cc:1034] running on GCE node
I20260927 23:02:31.590400 22211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:31.591626 22211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:31.592900 22211 hybrid_clock.cc:648] HybridClock initialized: now 1790550151592849 us; error 64 us; skew 500 ppm
I20260927 23:02:31.593292 22211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:31.596243 22211 webserver.cc:460] Webserver started at http://127.4.74.129:42847/ using document root <none> and password file <none>
I20260927 23:02:31.597055 22211 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:31.597201 22211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:31.597575 22211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:31.600906 22211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "794a7447ffe04d02a24742effa74c09a"
format_stamp: "Formatted at 2026-09-27 23:02:31 on dist-test-slave-5pd3"
server_key: "3ffb01b0b2de2856d23515b326602c6e"
server_key_iv: "1c0d01f1dd094efb8b26669d15db52da"
server_key_version: "encryptionkey@0"
I20260927 23:02:31.601840 22211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "794a7447ffe04d02a24742effa74c09a"
format_stamp: "Formatted at 2026-09-27 23:02:31 on dist-test-slave-5pd3"
server_key: "3ffb01b0b2de2856d23515b326602c6e"
server_key_iv: "1c0d01f1dd094efb8b26669d15db52da"
server_key_version: "encryptionkey@0"
I20260927 23:02:31.608083 22211 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260927 23:02:31.612388 22225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:31.614436 22211 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:02:31.614686 22211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "794a7447ffe04d02a24742effa74c09a"
format_stamp: "Formatted at 2026-09-27 23:02:31 on dist-test-slave-5pd3"
server_key: "3ffb01b0b2de2856d23515b326602c6e"
server_key_iv: "1c0d01f1dd094efb8b26669d15db52da"
server_key_version: "encryptionkey@0"
I20260927 23:02:31.614957 22211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:31.645243 22211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:31.646230 22211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:31.646559 22211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:31.647836 22211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:31.649670 22211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:31.649778 22211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:31.649863 22211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:31.649920 22211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:31.689409 22211 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:44617
I20260927 23:02:31.689615 22337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:44617 every 8 connection(s)
I20260927 23:02:31.690918 22211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:02:31.700289 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 22211
I20260927 23:02:31.700520 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 23:02:31.700986 4394 external_mini_cluster.cc:1428] Setting key 15d12b9a98f4027cf81f3f990c4a0644
I20260927 23:02:31.704489 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35175
--builtin_ntp_servers=127.4.74.148:36621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 23:02:31.707247 22340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:31.694252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:47757 (local address 127.4.74.190:35175)
0927 23:02:31.694843 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:31.694851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:31.696465 (+ 1614us) server_negotiation.cc:407] Connection header received
0927 23:02:31.698501 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:31.698508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:31.698612 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:31.698760 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:31.700394 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:31.701406 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:31.703714 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:31.704156 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:31.704267 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:31.706230 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:31.706235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:31.706239 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:31.706271 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:31.706325 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:31.706339 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:31.706418 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:31.706774 (+ 356us) server_negotiation.cc:299] Negotiation successful
0927 23:02:31.706886 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":271,"threads_started":1}
I20260927 23:02:31.708762 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:31.694987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35175 (local address 127.4.74.129:47757)
0927 23:02:31.696266 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:31.696313 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:31.698099 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:31.699064 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:31.699073 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:31.699183 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:31.700107 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:31.700145 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:31.701686 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:31.701691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:31.703483 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:31.703507 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:31.704439 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:31.704450 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:31.704825 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:31.705743 (+ 918us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 23:02:31.705749 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:31.705822 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 23:02:31.705947 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:31.706454 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:31.706468 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:31.706479 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 23:02:31.706702 (+ 223us) client_negotiation.cc:240] Negotiation successful
0927 23:02:31.707202 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":351,"threads_started":1}
I20260927 23:02:31.711475 22338 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35175
I20260927 23:02:31.711928 22338 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:31.712998 22338 heartbeater.cc:511] Master 127.4.74.190:35175 requested a full tablet report, sending...
I20260927 23:02:31.716225 22152 ts_manager.cc:194] Registered new tserver with Master: 794a7447ffe04d02a24742effa74c09a (127.4.74.129:44617)
I20260927 23:02:31.718880 22152 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.129:47757
W20260927 23:02:32.146756 22342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:32.147164 22342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:32.147274 22342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:32.156260 22342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:32.156411 22342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:32.156543 22342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:02:32.167409 22342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:32.169250 22342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:32.171423 22342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:32.189271 22347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:32.189397 22348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:32.189922 22350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:32.191347 22342 server_base.cc:1034] running on GCE node
I20260927 23:02:32.192196 22342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:32.193506 22342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:32.194772 22342 hybrid_clock.cc:648] HybridClock initialized: now 1790550152194689 us; error 70 us; skew 500 ppm
I20260927 23:02:32.195192 22342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:32.204568 22342 webserver.cc:460] Webserver started at http://127.4.74.130:33427/ using document root <none> and password file <none>
I20260927 23:02:32.205505 22342 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:32.205621 22342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:32.206017 22342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:32.209585 22342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "931c002170a04887917c58793fa9afa6"
format_stamp: "Formatted at 2026-09-27 23:02:32 on dist-test-slave-5pd3"
server_key: "10ad3d841b2cbf52f05a7059fab58bb9"
server_key_iv: "b66a0c2095d1895dca8a6c896ef4d3d3"
server_key_version: "encryptionkey@0"
I20260927 23:02:32.210566 22342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "931c002170a04887917c58793fa9afa6"
format_stamp: "Formatted at 2026-09-27 23:02:32 on dist-test-slave-5pd3"
server_key: "10ad3d841b2cbf52f05a7059fab58bb9"
server_key_iv: "b66a0c2095d1895dca8a6c896ef4d3d3"
server_key_version: "encryptionkey@0"
I20260927 23:02:32.217562 22342 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260927 23:02:32.222628 22356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:32.225325 22342 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260927 23:02:32.225594 22342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "931c002170a04887917c58793fa9afa6"
format_stamp: "Formatted at 2026-09-27 23:02:32 on dist-test-slave-5pd3"
server_key: "10ad3d841b2cbf52f05a7059fab58bb9"
server_key_iv: "b66a0c2095d1895dca8a6c896ef4d3d3"
server_key_version: "encryptionkey@0"
I20260927 23:02:32.225880 22342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:32.242969 22342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:32.244019 22342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:32.244379 22342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:32.245654 22342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:32.247401 22342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:32.247495 22342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:32.247673 22342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:32.247751 22342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:32.287681 22342 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:46451
I20260927 23:02:32.287904 22468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:46451 every 8 connection(s)
I20260927 23:02:32.289182 22342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:02:32.291766 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 22342
I20260927 23:02:32.291970 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 23:02:32.292450 4394 external_mini_cluster.cc:1428] Setting key 3a8717ae31069578da705a73d09fa193
I20260927 23:02:32.297052 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35175
--builtin_ntp_servers=127.4.74.148:36621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 23:02:32.314078 22472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.294269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35175 (local address 127.4.74.130:55109)
0927 23:02:32.295867 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:32.295931 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:32.298893 (+ 2962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:32.300304 (+ 1411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:32.300319 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:32.300476 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:32.301402 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:32.301429 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.305117 (+ 3688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.305127 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:32.307169 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:32.307195 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.309143 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.309151 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:32.309419 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.310463 (+ 1044us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 23:02:32.310472 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:32.310531 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 23:02:32.310656 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:32.311525 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:32.311536 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:32.311544 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 23:02:32.311875 (+ 331us) client_negotiation.cc:240] Negotiation successful
0927 23:02:32.312308 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":455,"threads_started":1}
I20260927 23:02:32.314596 22471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.293425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:55109 (local address 127.4.74.190:35175)
0927 23:02:32.293957 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:32.293964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:32.296181 (+ 2217us) server_negotiation.cc:407] Connection header received
0927 23:02:32.299406 (+ 3225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:32.299412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:32.299522 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:32.300035 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:32.301687 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.304580 (+ 2893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.307534 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.308717 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.309016 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.311095 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:32.311105 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:32.311114 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:32.311151 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:32.311226 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:32.311249 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:32.311369 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:32.312152 (+ 783us) server_negotiation.cc:299] Negotiation successful
0927 23:02:32.313466 (+ 1314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":275,"threads_started":1}
I20260927 23:02:32.316833 22469 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35175
I20260927 23:02:32.317217 22469 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:32.318699 22469 heartbeater.cc:511] Master 127.4.74.190:35175 requested a full tablet report, sending...
I20260927 23:02:32.322413 22152 ts_manager.cc:194] Registered new tserver with Master: 931c002170a04887917c58793fa9afa6 (127.4.74.130:46451)
I20260927 23:02:32.323489 22152 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.130:55109
W20260927 23:02:32.694222 22473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:32.694608 22473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:32.694772 22473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:32.703047 22473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:32.703236 22473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:32.703423 22473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:02:32.713383 22473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:35175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:32.714855 22473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:32.716832 22473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 23:02:32.722900 22338 heartbeater.cc:503] Master 127.4.74.190:35175 was elected leader, sending a full tablet report...
W20260927 23:02:32.732537 22478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:32.732642 22479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:32.732642 22481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:32.733908 22473 server_base.cc:1034] running on GCE node
I20260927 23:02:32.734736 22473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:32.736097 22473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:32.737443 22473 hybrid_clock.cc:648] HybridClock initialized: now 1790550152737352 us; error 74 us; skew 500 ppm
I20260927 23:02:32.737905 22473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:32.740940 22473 webserver.cc:460] Webserver started at http://127.4.74.131:33755/ using document root <none> and password file <none>
I20260927 23:02:32.741887 22473 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:32.741997 22473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:32.742441 22473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:32.746351 22473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "4aac40ba30d743f3a420ebbe827a0525"
format_stamp: "Formatted at 2026-09-27 23:02:32 on dist-test-slave-5pd3"
server_key: "d39918cac220c794600e4633bf5009bd"
server_key_iv: "730020bbacc6b517ab32d09d3da0ed20"
server_key_version: "encryptionkey@0"
I20260927 23:02:32.747444 22473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "4aac40ba30d743f3a420ebbe827a0525"
format_stamp: "Formatted at 2026-09-27 23:02:32 on dist-test-slave-5pd3"
server_key: "d39918cac220c794600e4633bf5009bd"
server_key_iv: "730020bbacc6b517ab32d09d3da0ed20"
server_key_version: "encryptionkey@0"
I20260927 23:02:32.755016 22473 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260927 23:02:32.761209 22487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:32.763446 22473 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260927 23:02:32.763715 22473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "4aac40ba30d743f3a420ebbe827a0525"
format_stamp: "Formatted at 2026-09-27 23:02:32 on dist-test-slave-5pd3"
server_key: "d39918cac220c794600e4633bf5009bd"
server_key_iv: "730020bbacc6b517ab32d09d3da0ed20"
server_key_version: "encryptionkey@0"
I20260927 23:02:32.763971 22473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:32.795660 22473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:32.796689 22473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:32.797093 22473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:32.798583 22473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:32.800567 22473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:32.800700 22473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:32.800818 22473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:32.800926 22473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:32.840073 22473 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:41889
I20260927 23:02:32.840219 22599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:41889 every 8 connection(s)
I20260927 23:02:32.841504 22473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:02:32.849412 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 22473
I20260927 23:02:32.849715 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 23:02:32.850283 4394 external_mini_cluster.cc:1428] Setting key f9b332e0e80aedbe4a246c19957a2397
I20260927 23:02:32.858155 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.844862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:56151 (local address 127.4.74.190:35175)
0927 23:02:32.845470 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:32.845477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:32.847101 (+ 1624us) server_negotiation.cc:407] Connection header received
0927 23:02:32.849406 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:32.849413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:32.849535 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:32.849708 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:32.851397 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.852533 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.854792 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.855238 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.855359 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.857167 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:32.857173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:32.857176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:32.857201 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:32.857255 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:32.857269 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:32.857353 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:32.857648 (+ 295us) server_negotiation.cc:299] Negotiation successful
0927 23:02:32.857781 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":290,"threads_started":1}
I20260927 23:02:32.859213 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.845479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:35175 (local address 127.4.74.131:56151)
0927 23:02:32.846863 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:32.846919 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:32.849018 (+ 2099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:32.850137 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:32.850164 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:32.850328 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:32.851171 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:32.851192 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.852726 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.852730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:32.854607 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:32.854620 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.855499 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.855505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:32.855866 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.856756 (+ 890us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 23:02:32.856759 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:32.856808 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 23:02:32.856907 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:32.857375 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:32.857384 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:32.857391 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 23:02:32.857564 (+ 173us) client_negotiation.cc:240] Negotiation successful
0927 23:02:32.857945 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":405,"threads_started":1}
I20260927 23:02:32.861220 22600 heartbeater.cc:347] Connected to a master server at 127.4.74.190:35175
I20260927 23:02:32.861625 22600 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:32.862514 22600 heartbeater.cc:511] Master 127.4.74.190:35175 requested a full tablet report, sending...
I20260927 23:02:32.864485 22152 ts_manager.cc:194] Registered new tserver with Master: 4aac40ba30d743f3a420ebbe827a0525 (127.4.74.131:41889)
I20260927 23:02:32.865821 22152 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.131:56151
I20260927 23:02:32.867443 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:02:32.880825 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.873412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.4.74.190:35175)
0927 23:02:32.873653 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:32.873659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:32.874301 (+ 642us) server_negotiation.cc:407] Connection header received
0927 23:02:32.874480 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:32.874485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:32.874575 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:32.874715 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:32.875307 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.876240 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.877968 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.878445 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.878577 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.879981 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:32.879987 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:32.879991 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:32.880028 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:32.880075 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:32.880089 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:32.880186 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:32.880372 (+ 186us) server_negotiation.cc:299] Negotiation successful
0927 23:02:32.880513 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260927 23:02:32.885804 22152 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:02:32.893151 22152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:02:32.920859 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.909032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:46451 (local address 127.0.0.1:52996)
0927 23:02:32.910266 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:32.910335 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:32.910590 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:32.911670 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:32.911679 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:32.911714 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:32.912111 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:32.912127 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.914558 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.914563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:32.917199 (+ 2636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:32.917216 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.919769 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.919777 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:32.919996 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.920078 (+ 82us) client_negotiation.cc:769] Sending connection context
0927 23:02:32.920212 (+ 134us) client_negotiation.cc:240] Negotiation successful
0927 23:02:32.920434 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"thread_start_us":648,"threads_started":1}
I20260927 23:02:32.920882 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.910847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:41889 (local address 127.0.0.1:47384)
0927 23:02:32.911506 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:32.911527 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:32.911667 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:32.912975 (+ 1308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:32.912980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:32.913008 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:32.913520 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:32.913533 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.915702 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.915708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:32.918039 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:32.918054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.920190 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.920200 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:32.920374 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.920404 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 23:02:32.920461 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 23:02:32.920571 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":339,"threads_started":1}
I20260927 23:02:32.922076 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.911223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47384 (local address 127.4.74.131:41889)
0927 23:02:32.912337 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:32.912347 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:32.912386 (+ 39us) server_negotiation.cc:407] Connection header received
0927 23:02:32.912489 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:32.912496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:32.912756 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:32.913739 (+ 983us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:32.914023 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.915518 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.918348 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.919945 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.921053 (+ 1108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.921434 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:32.921521 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 23:02:32.921705 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":703,"threads_started":1}
I20260927 23:02:32.922322 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.908994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52996 (local address 127.4.74.130:46451)
0927 23:02:32.909622 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:32.909646 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:32.910546 (+ 900us) server_negotiation.cc:407] Connection header received
0927 23:02:32.910872 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:32.910879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:32.911150 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:32.911361 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:32.912310 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.914381 (+ 2071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.917591 (+ 3210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.919335 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.919497 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.921692 (+ 2195us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:32.921761 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 23:02:32.921943 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":317,"threads_started":1}
I20260927 23:02:32.929114 22535 tablet_service.cc:1467] Processing CreateTablet for tablet 93723845697241df9a2ebb2254cee8fc (DEFAULT_TABLE table=test-table [id=4eb5b6b9fd5342a294d0d71dffc76a05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:32.929962 22404 tablet_service.cc:1467] Processing CreateTablet for tablet 93723845697241df9a2ebb2254cee8fc (DEFAULT_TABLE table=test-table [id=4eb5b6b9fd5342a294d0d71dffc76a05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:32.931325 22535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93723845697241df9a2ebb2254cee8fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:32.932313 22404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93723845697241df9a2ebb2254cee8fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:32.932634 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.912348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44617 (local address 127.0.0.1:50190)
0927 23:02:32.921211 (+ 8863us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:32.921234 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:32.921350 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:32.924769 (+ 3419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:32.924775 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:32.924805 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:32.925034 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:32.925049 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.927058 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.927070 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:32.929172 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:32.929206 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.931620 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.931629 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:32.931881 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.931923 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 23:02:32.932059 (+ 136us) client_negotiation.cc:240] Negotiation successful
0927 23:02:32.932260 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8715,"thread_start_us":12196,"threads_started":1}
I20260927 23:02:32.932713 22617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.912059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50190 (local address 127.4.74.129:44617)
0927 23:02:32.924168 (+ 12109us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:32.924176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:32.924205 (+ 29us) server_negotiation.cc:407] Connection header received
0927 23:02:32.924296 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:32.924303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:32.924536 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:32.924778 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:32.925236 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.926829 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.929473 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:32.931056 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:32.931248 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:32.931589 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:32.932153 (+ 564us) server_negotiation.cc:299] Negotiation successful
0927 23:02:32.932332 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11922,"thread_start_us":11836,"threads_started":1}
I20260927 23:02:32.937453 22273 tablet_service.cc:1467] Processing CreateTablet for tablet 93723845697241df9a2ebb2254cee8fc (DEFAULT_TABLE table=test-table [id=4eb5b6b9fd5342a294d0d71dffc76a05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:32.939074 22273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93723845697241df9a2ebb2254cee8fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:32.947965 22619 tablet_bootstrap.cc:492] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: Bootstrap starting.
I20260927 23:02:32.952646 22620 tablet_bootstrap.cc:492] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: Bootstrap starting.
I20260927 23:02:32.955241 22621 tablet_bootstrap.cc:492] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a: Bootstrap starting.
I20260927 23:02:32.956874 22619 tablet_bootstrap.cc:654] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:32.957070 22620 tablet_bootstrap.cc:654] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:32.958220 22620 log.cc:824] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:32.958751 22619 log.cc:824] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:32.960376 22621 tablet_bootstrap.cc:654] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:32.961804 22620 tablet_bootstrap.cc:492] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: No bootstrap required, opened a new log
I20260927 23:02:32.961997 22621 log.cc:824] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:32.962129 22620 ts_tablet_manager.cc:1397] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260927 23:02:32.963845 22619 tablet_bootstrap.cc:492] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: No bootstrap required, opened a new log
I20260927 23:02:32.964259 22619 ts_tablet_manager.cc:1397] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.003s
I20260927 23:02:32.966253 22621 tablet_bootstrap.cc:492] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a: No bootstrap required, opened a new log
I20260927 23:02:32.966591 22621 ts_tablet_manager.cc:1397] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260927 23:02:32.971436 22620 raft_consensus.cc:348] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931c002170a04887917c58793fa9afa6" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 46451 } } peers { permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 } } peers { permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 } }
I20260927 23:02:32.971967 22620 raft_consensus.cc:374] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:32.972062 22620 raft_consensus.cc:729] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931c002170a04887917c58793fa9afa6, State: Initialized, Role: FOLLOWER
I20260927 23:02:32.972770 22620 consensus_queue.cc:260] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931c002170a04887917c58793fa9afa6" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 46451 } } peers { permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 } } peers { permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 } }
I20260927 23:02:32.974201 22469 heartbeater.cc:503] Master 127.4.74.190:35175 was elected leader, sending a full tablet report...
I20260927 23:02:32.974668 22620 ts_tablet_manager.cc:1428] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260927 23:02:32.977994 22621 raft_consensus.cc:348] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931c002170a04887917c58793fa9afa6" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 46451 } } peers { permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 } } peers { permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 } }
I20260927 23:02:32.978583 22621 raft_consensus.cc:374] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:32.978690 22621 raft_consensus.cc:729] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 794a7447ffe04d02a24742effa74c09a, State: Initialized, Role: FOLLOWER
I20260927 23:02:32.978420 22619 raft_consensus.cc:348] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931c002170a04887917c58793fa9afa6" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 46451 } } peers { permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 } } peers { permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 } }
I20260927 23:02:32.979051 22619 raft_consensus.cc:374] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:32.979171 22619 raft_consensus.cc:729] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aac40ba30d743f3a420ebbe827a0525, State: Initialized, Role: FOLLOWER
I20260927 23:02:32.979807 22621 consensus_queue.cc:260] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931c002170a04887917c58793fa9afa6" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 46451 } } peers { permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 } } peers { permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 } }
I20260927 23:02:32.979985 22619 consensus_queue.cc:260] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931c002170a04887917c58793fa9afa6" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 46451 } } peers { permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 } } peers { permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 } }
I20260927 23:02:32.983266 22600 heartbeater.cc:503] Master 127.4.74.190:35175 was elected leader, sending a full tablet report...
I20260927 23:02:32.983729 22621 ts_tablet_manager.cc:1428] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a: Time spent starting tablet: real 0.017s user 0.006s sys 0.013s
I20260927 23:02:32.984591 22619 ts_tablet_manager.cc:1428] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: Time spent starting tablet: real 0.020s user 0.013s sys 0.004s
I20260927 23:02:32.996393 22625 raft_consensus.cc:482] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:02:32.996906 22625 raft_consensus.cc:504] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931c002170a04887917c58793fa9afa6" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 46451 } } peers { permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 } } peers { permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 } }
I20260927 23:02:32.999414 22625 leader_election.cc:288] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4aac40ba30d743f3a420ebbe827a0525 (127.4.74.131:41889), 794a7447ffe04d02a24742effa74c09a (127.4.74.129:44617)
I20260927 23:02:33.009362 22617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.001019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:56097 (local address 127.4.74.129:44617)
0927 23:02:33.001318 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:33.001326 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:33.001746 (+ 420us) server_negotiation.cc:407] Connection header received
0927 23:02:33.001958 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:33.001966 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:33.002075 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:33.002223 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:33.002863 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.004106 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.007013 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.008118 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.008374 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:33.008470 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:33.008833 (+ 363us) server_negotiation.cc:299] Negotiation successful
0927 23:02:33.009026 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260927 23:02:33.009718 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.999900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:41737 (local address 127.4.74.131:41889)
0927 23:02:33.000220 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:33.000229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:33.000783 (+ 554us) server_negotiation.cc:407] Connection header received
0927 23:02:33.001187 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:33.001198 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:33.001306 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:33.001381 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:33.002317 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.003366 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.006501 (+ 3135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.008162 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.008398 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:33.008480 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:33.009199 (+ 719us) server_negotiation.cc:299] Negotiation successful
0927 23:02:33.009382 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260927 23:02:33.009425 22629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.000684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44617 (local address 127.4.74.130:56097)
0927 23:02:33.001614 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:33.001631 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:33.001774 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:33.002250 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:33.002258 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:33.002289 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:33.002682 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:33.002696 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.004286 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.004292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:33.006727 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:33.006740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.008378 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.008386 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:33.008627 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:33.008659 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 23:02:33.008795 (+ 136us) client_negotiation.cc:240] Negotiation successful
0927 23:02:33.009014 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":636,"threads_started":1}
I20260927 23:02:33.010473 22628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:32.999725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:41889 (local address 127.4.74.130:41737)
0927 23:02:33.000612 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:33.000641 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:33.000973 (+ 332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:33.001650 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:33.001659 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:33.001693 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:33.002123 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:33.002133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.003510 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.003519 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:33.006158 (+ 2639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:33.006188 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.008699 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.008708 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:33.008984 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:33.009027 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 23:02:33.009808 (+ 781us) client_negotiation.cc:240] Negotiation successful
0927 23:02:33.010020 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":7,"thread_start_us":447,"threads_started":1}
I20260927 23:02:33.011691 22293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93723845697241df9a2ebb2254cee8fc" candidate_uuid: "931c002170a04887917c58793fa9afa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "794a7447ffe04d02a24742effa74c09a" is_pre_election: true
I20260927 23:02:33.011981 22555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93723845697241df9a2ebb2254cee8fc" candidate_uuid: "931c002170a04887917c58793fa9afa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aac40ba30d743f3a420ebbe827a0525" is_pre_election: true
I20260927 23:02:33.012550 22555 raft_consensus.cc:2454] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931c002170a04887917c58793fa9afa6 in term 0.
I20260927 23:02:33.012557 22293 raft_consensus.cc:2454] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931c002170a04887917c58793fa9afa6 in term 0.
I20260927 23:02:33.013446 22357 leader_election.cc:302] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aac40ba30d743f3a420ebbe827a0525, 931c002170a04887917c58793fa9afa6; no voters:
I20260927 23:02:33.014045 22625 raft_consensus.cc:2781] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:02:33.014284 22625 raft_consensus.cc:482] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:02:33.014424 22625 raft_consensus.cc:3037] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:33.017206 22625 raft_consensus.cc:504] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931c002170a04887917c58793fa9afa6" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 46451 } } peers { permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 } } peers { permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 } }
I20260927 23:02:33.018226 22625 leader_election.cc:288] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [CANDIDATE]: Term 1 election: Requested vote from peers 4aac40ba30d743f3a420ebbe827a0525 (127.4.74.131:41889), 794a7447ffe04d02a24742effa74c09a (127.4.74.129:44617)
I20260927 23:02:33.019045 22555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93723845697241df9a2ebb2254cee8fc" candidate_uuid: "931c002170a04887917c58793fa9afa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aac40ba30d743f3a420ebbe827a0525"
I20260927 23:02:33.019080 22293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93723845697241df9a2ebb2254cee8fc" candidate_uuid: "931c002170a04887917c58793fa9afa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "794a7447ffe04d02a24742effa74c09a"
I20260927 23:02:33.019359 22555 raft_consensus.cc:3037] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:33.019359 22293 raft_consensus.cc:3037] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:33.023303 22293 raft_consensus.cc:2454] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931c002170a04887917c58793fa9afa6 in term 1.
I20260927 23:02:33.023303 22555 raft_consensus.cc:2454] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931c002170a04887917c58793fa9afa6 in term 1.
I20260927 23:02:33.024016 22357 leader_election.cc:302] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 794a7447ffe04d02a24742effa74c09a, 931c002170a04887917c58793fa9afa6; no voters:
I20260927 23:02:33.024582 22625 raft_consensus.cc:2781] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:33.025178 22625 raft_consensus.cc:686] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 1 LEADER]: Becoming Leader. State: Replica: 931c002170a04887917c58793fa9afa6, State: Running, Role: LEADER
I20260927 23:02:33.025811 22625 consensus_queue.cc:237] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931c002170a04887917c58793fa9afa6" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 46451 } } peers { permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 } } peers { permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 } }
I20260927 23:02:33.032418 22152 catalog_manager.cc:5495] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 931c002170a04887917c58793fa9afa6 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "931c002170a04887917c58793fa9afa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931c002170a04887917c58793fa9afa6" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 46451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 } health_report { overall_health: UNKNOWN } } }
W20260927 23:02:33.041919 22470 tablet.cc:2369] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:33.057874 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.051288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53008 (local address 127.4.74.130:46451)
0927 23:02:33.051494 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:33.051498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:33.051521 (+ 23us) server_negotiation.cc:407] Connection header received
0927 23:02:33.051695 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:33.051699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:33.051806 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:33.051973 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:02:33.052522 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.053304 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.054900 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.055362 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.055485 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:33.056464 (+ 979us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:02:33.057070 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:02:33.057217 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:33.057497 (+ 280us) server_negotiation.cc:299] Negotiation successful
0927 23:02:33.057584 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 23:02:33.068593 22555 raft_consensus.cc:1261] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525 [term 1 FOLLOWER]: Refusing update from remote peer 931c002170a04887917c58793fa9afa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:33.068586 22293 raft_consensus.cc:1261] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a [term 1 FOLLOWER]: Refusing update from remote peer 931c002170a04887917c58793fa9afa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:33.070254 22630 consensus_queue.cc:1035] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aac40ba30d743f3a420ebbe827a0525" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:33.070751 22625 consensus_queue.cc:1035] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "794a7447ffe04d02a24742effa74c09a" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:33.085741 22633 mvcc.cc:205] Tried to move back new op lower bound from 7334093426954223616 to 7334093426798800896. Current Snapshot: MvccSnapshot[applied={T|T < 7334093426954223616}]
I20260927 23:02:33.086546 22635 mvcc.cc:205] Tried to move back new op lower bound from 7334093426954223616 to 7334093426798800896. Current Snapshot: MvccSnapshot[applied={T|T < 7334093426954223616}]
I20260927 23:02:33.104661 22151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:33.105057 22151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:33.110294 22151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5504470534f94108a83b205752ddcc63: Sending DeleteTablet for 3 replicas of tablet 93723845697241df9a2ebb2254cee8fc
I20260927 23:02:33.112098 22535 tablet_service.cc:1514] Processing DeleteTablet for tablet 93723845697241df9a2ebb2254cee8fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:33 UTC) from {username='slave'} at 127.0.0.1:47384
I20260927 23:02:33.112143 22273 tablet_service.cc:1514] Processing DeleteTablet for tablet 93723845697241df9a2ebb2254cee8fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:33 UTC) from {username='slave'} at 127.0.0.1:50190
I20260927 23:02:33.112151 22404 tablet_service.cc:1514] Processing DeleteTablet for tablet 93723845697241df9a2ebb2254cee8fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:33 UTC) from {username='slave'} at 127.0.0.1:52996
I20260927 23:02:33.114872 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 22211
I20260927 23:02:33.125005 22645 tablet_replica.cc:330] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a: stopping tablet replica
I20260927 23:02:33.125679 22645 raft_consensus.cc:2229] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:33.126407 22645 raft_consensus.cc:2258] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:33.131973 22643 tablet_replica.cc:330] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: stopping tablet replica
I20260927 23:02:33.132689 22643 raft_consensus.cc:2229] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:33.133415 22643 raft_consensus.cc:2258] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:33.134282 22645 ts_tablet_manager.cc:1905] T 93723845697241df9a2ebb2254cee8fc P 794a7447ffe04d02a24742effa74c09a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:33.135946 22643 ts_tablet_manager.cc:1905] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:33.136843 22617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.116583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.4.74.129:44617)
0927 23:02:33.116936 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:33.116947 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:33.123072 (+ 6125us) server_negotiation.cc:407] Connection header received
0927 23:02:33.124513 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:33.124523 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:33.124706 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:33.124810 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:33.126077 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.127774 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.131961 (+ 4187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.132749 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.132887 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:33.134723 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:33.134736 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:33.134756 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:33.135669 (+ 913us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:33.135755 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:33.135788 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:33.135892 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:33.136181 (+ 289us) server_negotiation.cc:299] Negotiation successful
0927 23:02:33.136329 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260927 23:02:33.137786 22644 tablet_replica.cc:330] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: stopping tablet replica
I20260927 23:02:33.138182 22333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:33.138456 22644 raft_consensus.cc:2229] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:02:33.139317 22644 raft_consensus.cc:2258] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:33.142642 22644 ts_tablet_manager.cc:1905] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:33.146119 22643 ts_tablet_manager.cc:1918] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:33.146344 22643 log.cc:1196] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/93723845697241df9a2ebb2254cee8fc
I20260927 23:02:33.146968 22643 ts_tablet_manager.cc:1939] T 93723845697241df9a2ebb2254cee8fc P 4aac40ba30d743f3a420ebbe827a0525: Deleting consensus metadata
I20260927 23:02:33.149184 22136 catalog_manager.cc:4841] TS 4aac40ba30d743f3a420ebbe827a0525 (127.4.74.131:41889): tablet 93723845697241df9a2ebb2254cee8fc (table test-table [id=4eb5b6b9fd5342a294d0d71dffc76a05]) successfully deleted
I20260927 23:02:33.154740 22644 ts_tablet_manager.cc:1918] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:33.155000 22644 log.cc:1196] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/93723845697241df9a2ebb2254cee8fc
I20260927 23:02:33.155690 22644 ts_tablet_manager.cc:1939] T 93723845697241df9a2ebb2254cee8fc P 931c002170a04887917c58793fa9afa6: Deleting consensus metadata
I20260927 23:02:33.158253 22136 catalog_manager.cc:4841] TS 931c002170a04887917c58793fa9afa6 (127.4.74.130:46451): tablet 93723845697241df9a2ebb2254cee8fc (table test-table [id=4eb5b6b9fd5342a294d0d71dffc76a05]) successfully deleted
I20260927 23:02:33.245647 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 22211
W20260927 23:02:33.265692 22136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.74.129:44617 (error 108)
I20260927 23:02:33.266273 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 22342
I20260927 23:02:33.268311 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.267672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44617 (local address 127.0.0.1:50198)
0927 23:02:33.267916 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:33.268038 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260927 23:02:33.268611 22136 catalog_manager.cc:4584] TS 794a7447ffe04d02a24742effa74c09a (127.4.74.129:44617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93723845697241df9a2ebb2254cee8fc: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44617: connect: Connection refused (error 111)
I20260927 23:02:33.276158 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.267333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53022 (local address 127.4.74.130:46451)
0927 23:02:33.267541 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:33.267585 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:33.267949 (+ 364us) server_negotiation.cc:407] Connection header received
0927 23:02:33.268060 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:33.268066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:33.268218 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:33.268345 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:33.269010 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.270365 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.272023 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.272753 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.272905 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:33.274046 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:33.274060 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:33.274083 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:33.275121 (+ 1038us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:33.275204 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:33.275236 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:33.275377 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:33.275515 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 23:02:33.275702 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 23:02:33.277073 22464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:33.322324 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.321811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44617 (local address 127.0.0.1:50206)
0927 23:02:33.322066 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:33.322153 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260927 23:02:33.323378 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.323038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44617 (local address 127.0.0.1:50222)
0927 23:02:33.323196 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:33.323260 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260927 23:02:33.384317 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.383846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44617 (local address 127.0.0.1:50236)
0927 23:02:33.384068 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:33.384160 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260927 23:02:33.385314 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.384983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44617 (local address 127.0.0.1:50250)
0927 23:02:33.385119 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:33.385189 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260927 23:02:33.409437 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 22342
I20260927 23:02:33.431072 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 22473
I20260927 23:02:33.438853 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.431985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47388 (local address 127.4.74.131:41889)
0927 23:02:33.432194 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:33.432199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:33.432243 (+ 44us) server_negotiation.cc:407] Connection header received
0927 23:02:33.432320 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:33.432324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:33.432452 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:33.432551 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:33.433011 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.433844 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.435210 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:33.435817 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:33.435944 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:33.437218 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:33.437227 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:33.437247 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:33.438075 (+ 828us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:33.438137 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:33.438167 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:33.438271 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:33.438420 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 23:02:33.438547 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260927 23:02:33.439724 22595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:33.466513 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.465812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44617 (local address 127.0.0.1:50260)
0927 23:02:33.466166 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:33.466300 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260927 23:02:33.468096 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.467536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44617 (local address 127.0.0.1:50270)
0927 23:02:33.467819 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:33.467931 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260927 23:02:33.533953 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 22473
I20260927 23:02:33.553658 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 22121
I20260927 23:02:33.555117 22182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:33.668987 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:33.667937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44617 (local address 127.0.0.1:50274)
0927 23:02:33.668455 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:33.668639 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260927 23:02:33.669332 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 22121
2026-09-27T23:02:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3256 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:02:33 dist-test-slave-5pd3 krb5kdc[22654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:02:33 dist-test-slave-5pd3 krb5kdc[22654](info): set up 2 sockets
Sep 27 23:02:33 dist-test-slave-5pd3 krb5kdc[22654](info): commencing operation
krb5kdc: starting...
W20260927 23:02:34.839977 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:02:35 dist-test-slave-5pd3 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:02:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:02:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:02:35.141455 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:39545
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:38167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:39545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:35.495692 22667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:35.496048 22667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:35.496186 22667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:35.503983 22667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:35.504117 22667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:35.504165 22667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:35.504293 22667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:35.504370 22667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:35.514988 22667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:39545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:39545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:35.516836 22667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:35.518232 22667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:35.531944 22673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:35.532043 22672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:35.532648 22675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:35.533149 22667 server_base.cc:1034] running on GCE node
I20260927 23:02:35.534042 22667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:35.535789 22667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:35.537086 22667 hybrid_clock.cc:648] HybridClock initialized: now 1790550155537020 us; error 58 us; skew 500 ppm
I20260927 23:02:35.537482 22667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:35 dist-test-slave-5pd3 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550155, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:35.541546 22667 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:02:35.543468 22667 webserver.cc:460] Webserver started at http://127.4.74.190:44831/ using document root <none> and password file <none>
I20260927 23:02:35.544322 22667 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:35.544489 22667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:35.544878 22667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:35.548395 22667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "cf9ecd1cb58847829f47f6c7a7585d6a"
format_stamp: "Formatted at 2026-09-27 23:02:35 on dist-test-slave-5pd3"
server_key: "fa8a172dc8356e8dd200bd743ea85c29"
server_key_iv: "26af830aa474f1a8abb62f4a47bba8e0"
server_key_version: "encryptionkey@0"
I20260927 23:02:35.549281 22667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "cf9ecd1cb58847829f47f6c7a7585d6a"
format_stamp: "Formatted at 2026-09-27 23:02:35 on dist-test-slave-5pd3"
server_key: "fa8a172dc8356e8dd200bd743ea85c29"
server_key_iv: "26af830aa474f1a8abb62f4a47bba8e0"
server_key_version: "encryptionkey@0"
I20260927 23:02:35.555384 22667 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260927 23:02:35.559661 22682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:35.561465 22667 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:02:35.561722 22667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "cf9ecd1cb58847829f47f6c7a7585d6a"
format_stamp: "Formatted at 2026-09-27 23:02:35 on dist-test-slave-5pd3"
server_key: "fa8a172dc8356e8dd200bd743ea85c29"
server_key_iv: "26af830aa474f1a8abb62f4a47bba8e0"
server_key_version: "encryptionkey@0"
I20260927 23:02:35.561962 22667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:35.578394 22667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:35.583233 22667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:35.583580 22667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:35.605751 22667 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:39545
I20260927 23:02:35.605852 22733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:39545 every 8 connection(s)
I20260927 23:02:35.607288 22667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:35.612247 22734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:35.613693 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 22667
I20260927 23:02:35.613933 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:02:35.614625 4394 external_mini_cluster.cc:1428] Setting key d0a03d07e21f44a7f82a975e14827603
I20260927 23:02:35.632088 22734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a: Bootstrap starting.
Sep 27 23:02:35 dist-test-slave-5pd3 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:35.637820 22734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:35.639739 22734 log.cc:824] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:35.644496 22734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a: No bootstrap required, opened a new log
I20260927 23:02:35.647035 22737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:35.617159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50604 (local address 127.4.74.190:39545)
0927 23:02:35.618164 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:35.618177 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:35.618234 (+ 57us) server_negotiation.cc:407] Connection header received
0927 23:02:35.620378 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:35.620405 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:35.620908 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:35.621334 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:35.624026 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:35.625467 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:35.627991 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:35.628842 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:35.629014 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:35.635750 (+ 6736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:35.635767 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:35.635795 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:35.635892 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:35.641063 (+ 5171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:35.642206 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:35.642217 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:35.642227 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:35.642507 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:35.643039 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:35.643045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:35.643048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:35.643911 (+ 863us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:35.644304 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:35.644766 (+ 462us) server_negotiation.cc:299] Negotiation successful
0927 23:02:35.645375 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":494,"threads_started":1}
I20260927 23:02:35.655342 22734 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf9ecd1cb58847829f47f6c7a7585d6a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39545 } }
I20260927 23:02:35.655994 22734 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:35.656095 22734 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9ecd1cb58847829f47f6c7a7585d6a, State: Initialized, Role: FOLLOWER
I20260927 23:02:35.656764 22734 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf9ecd1cb58847829f47f6c7a7585d6a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39545 } }
I20260927 23:02:35.657078 22734 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:35.657255 22734 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:35.657488 22734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:35.660254 22734 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf9ecd1cb58847829f47f6c7a7585d6a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39545 } }
I20260927 23:02:35.661001 22734 leader_election.cc:302] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf9ecd1cb58847829f47f6c7a7585d6a; no voters:
I20260927 23:02:35.661908 22734 leader_election.cc:288] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:02:35.662062 22739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:35.662963 22739 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [term 1 LEADER]: Becoming Leader. State: Replica: cf9ecd1cb58847829f47f6c7a7585d6a, State: Running, Role: LEADER
I20260927 23:02:35.663710 22739 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf9ecd1cb58847829f47f6c7a7585d6a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39545 } }
I20260927 23:02:35.664882 22734 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:35.667508 22741 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf9ecd1cb58847829f47f6c7a7585d6a. Latest consensus state: current_term: 1 leader_uuid: "cf9ecd1cb58847829f47f6c7a7585d6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf9ecd1cb58847829f47f6c7a7585d6a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39545 } } }
I20260927 23:02:35.668430 22741 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:35.668062 22740 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf9ecd1cb58847829f47f6c7a7585d6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf9ecd1cb58847829f47f6c7a7585d6a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 39545 } } }
I20260927 23:02:35.668731 22740 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:35.672959 22746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:35.678596 22746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:35.690321 22746 catalog_manager.cc:1347] Generated new cluster ID: 1715391d1849420db7c554a029ee65c8
I20260927 23:02:35.690529 22746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:35.716965 22746 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:02:35.718880 22746 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:35.734180 22746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a: Generated new TSK 0
I20260927 23:02:35.735323 22746 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 23:02:35.824182 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39545
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:38167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:36.205983 22760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:36.206326 22760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:36.206437 22760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:36.214864 22760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:36.215025 22760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:36.215153 22760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:02:36.225826 22760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:36.227146 22760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:36.228734 22760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:36.244570 22766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:36.244632 22768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:36.245010 22765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:36.246281 22760 server_base.cc:1034] running on GCE node
I20260927 23:02:36.247282 22760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:36.248759 22760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:36.250053 22760 hybrid_clock.cc:648] HybridClock initialized: now 1790550156249982 us; error 76 us; skew 500 ppm
I20260927 23:02:36.250481 22760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:36 dist-test-slave-5pd3 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550156, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:36.254822 22760 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:02:36.257174 22760 webserver.cc:460] Webserver started at http://127.4.74.129:33127/ using document root <none> and password file <none>
I20260927 23:02:36.258601 22760 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:36.258831 22760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:36.259265 22760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:36.264077 22760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "08f44293698a4c2b8b7e49d1f28c70b0"
format_stamp: "Formatted at 2026-09-27 23:02:36 on dist-test-slave-5pd3"
server_key: "ab92bb9eb2d01f2ad93c88cf65c896e5"
server_key_iv: "9695d2da20349d66ff43c11494d178fe"
server_key_version: "encryptionkey@0"
I20260927 23:02:36.265477 22760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "08f44293698a4c2b8b7e49d1f28c70b0"
format_stamp: "Formatted at 2026-09-27 23:02:36 on dist-test-slave-5pd3"
server_key: "ab92bb9eb2d01f2ad93c88cf65c896e5"
server_key_iv: "9695d2da20349d66ff43c11494d178fe"
server_key_version: "encryptionkey@0"
I20260927 23:02:36.273378 22760 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260927 23:02:36.278900 22775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:36.281337 22760 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260927 23:02:36.281664 22760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "08f44293698a4c2b8b7e49d1f28c70b0"
format_stamp: "Formatted at 2026-09-27 23:02:36 on dist-test-slave-5pd3"
server_key: "ab92bb9eb2d01f2ad93c88cf65c896e5"
server_key_iv: "9695d2da20349d66ff43c11494d178fe"
server_key_version: "encryptionkey@0"
I20260927 23:02:36.281981 22760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:36.298437 22760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:36.303843 22760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:36.304229 22760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:36.305546 22760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:36.307390 22760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:36.307515 22760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:36.307734 22760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:36.307813 22760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:36.352262 22760 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:41867
I20260927 23:02:36.352329 22887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:41867 every 8 connection(s)
I20260927 23:02:36.353884 22760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:02:36.360512 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 22760
I20260927 23:02:36.360738 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 23:02:36.361363 4394 external_mini_cluster.cc:1428] Setting key 81b891b498fa3500f316a2e54fe2bccf
Sep 27 23:02:36 dist-test-slave-5pd3 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550156, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:36.381204 22890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:36.357179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:34357 (local address 127.4.74.190:39545)
0927 23:02:36.357757 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:36.357766 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:36.359067 (+ 1301us) server_negotiation.cc:407] Connection header received
0927 23:02:36.361898 (+ 2831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:36.361905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:36.362010 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:36.362206 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:36.364739 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:36.365716 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:36.367839 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:36.368290 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:36.368444 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:36.373443 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:36.373449 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:36.373452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:36.373496 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:36.376767 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:36.378107 (+ 1340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:36.378113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:36.378116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:36.378296 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:36.378917 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:36.378922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:36.378925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:36.379300 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:36.379439 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:36.380615 (+ 1176us) server_negotiation.cc:299] Negotiation successful
0927 23:02:36.380867 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":260,"threads_started":1}
I20260927 23:02:36.382726 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:36.357882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39545 (local address 127.4.74.129:34357)
0927 23:02:36.358784 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:36.358860 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:36.361428 (+ 2568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:36.362602 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:36.362617 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:36.363658 (+ 1041us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:36.364485 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:36.364510 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:36.365917 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:36.365922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:36.367662 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:36.367680 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:36.368570 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:36.368580 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:36.368889 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:36.369805 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:02:36.369847 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:36.373068 (+ 3221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:36.377011 (+ 3943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:36.377026 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:36.377049 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:36.377848 (+ 799us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:36.378457 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:36.378465 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:36.378472 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:36.378753 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:36.379452 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:36.379464 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:36.380062 (+ 598us) client_negotiation.cc:769] Sending connection context
0927 23:02:36.380436 (+ 374us) client_negotiation.cc:240] Negotiation successful
0927 23:02:36.381138 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":327,"threads_started":1}
I20260927 23:02:36.384917 22888 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39545
I20260927 23:02:36.385363 22888 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:36.386338 22888 heartbeater.cc:511] Master 127.4.74.190:39545 requested a full tablet report, sending...
I20260927 23:02:36.389070 22699 ts_manager.cc:194] Registered new tserver with Master: 08f44293698a4c2b8b7e49d1f28c70b0 (127.4.74.129:41867)
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 23:02:36.390928 22699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:34357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 23:02:36.424983 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39545
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:38167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:36.786780 22894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:36.787127 22894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:36.787300 22894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:36.795274 22894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:36.795517 22894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:36.795827 22894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:02:36.806994 22894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:36.808833 22894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:36.810410 22894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:36.828116 22899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:36.828694 22900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:36.828744 22902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:36.830093 22894 server_base.cc:1034] running on GCE node
I20260927 23:02:36.830924 22894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:36.832772 22894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:36.834251 22894 hybrid_clock.cc:648] HybridClock initialized: now 1790550156834216 us; error 69 us; skew 500 ppm
I20260927 23:02:36.834700 22894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:36 dist-test-slave-5pd3 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550156, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:36.838976 22894 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:02:36.840910 22894 webserver.cc:460] Webserver started at http://127.4.74.130:38263/ using document root <none> and password file <none>
I20260927 23:02:36.841753 22894 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:36.841920 22894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:36.842265 22894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:36.845786 22894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "89271be385e54c0e84987413f92977ef"
format_stamp: "Formatted at 2026-09-27 23:02:36 on dist-test-slave-5pd3"
server_key: "6a407d3315fe97aeb51ff339dfb107f1"
server_key_iv: "7d36f1bfa0b2fee8bde8d0294835abc6"
server_key_version: "encryptionkey@0"
I20260927 23:02:36.846773 22894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "89271be385e54c0e84987413f92977ef"
format_stamp: "Formatted at 2026-09-27 23:02:36 on dist-test-slave-5pd3"
server_key: "6a407d3315fe97aeb51ff339dfb107f1"
server_key_iv: "7d36f1bfa0b2fee8bde8d0294835abc6"
server_key_version: "encryptionkey@0"
I20260927 23:02:36.853976 22894 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260927 23:02:36.858821 22909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:36.861068 22894 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260927 23:02:36.861351 22894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "89271be385e54c0e84987413f92977ef"
format_stamp: "Formatted at 2026-09-27 23:02:36 on dist-test-slave-5pd3"
server_key: "6a407d3315fe97aeb51ff339dfb107f1"
server_key_iv: "7d36f1bfa0b2fee8bde8d0294835abc6"
server_key_version: "encryptionkey@0"
I20260927 23:02:36.861613 22894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:36.882426 22894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:36.888233 22894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:36.888602 22894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:36.889984 22894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:36.891863 22894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:36.891997 22894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:36.892175 22894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:36.892257 22894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:36.936937 22894 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:40149
I20260927 23:02:36.937064 23021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:40149 every 8 connection(s)
I20260927 23:02:36.938546 22894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:02:36.944531 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 22894
I20260927 23:02:36.944767 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 23:02:36.945322 4394 external_mini_cluster.cc:1428] Setting key 406a57193fd4bd849f35d913f59b2ddb
Sep 27 23:02:36 dist-test-slave-5pd3 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550156, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:36.967931 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:36.942346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:45915 (local address 127.4.74.190:39545)
0927 23:02:36.943003 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:36.943009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:36.944390 (+ 1381us) server_negotiation.cc:407] Connection header received
0927 23:02:36.947405 (+ 3015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:36.947410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:36.947506 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:36.947738 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:36.950725 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:36.951682 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:36.953768 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:36.954309 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:36.954469 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:36.959263 (+ 4794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:36.959270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:36.959274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:36.959323 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:36.962539 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:36.963670 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:36.963679 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:36.963683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:36.963853 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:36.964425 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:36.964429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:36.964431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:36.964807 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:36.965006 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:36.967118 (+ 2112us) server_negotiation.cc:299] Negotiation successful
0927 23:02:36.967518 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":334,"threads_started":1}
I20260927 23:02:36.969259 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:36.943009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39545 (local address 127.4.74.130:45915)
0927 23:02:36.944121 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:36.944201 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:36.946932 (+ 2731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:36.948191 (+ 1259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:36.948210 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:36.949484 (+ 1274us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:36.950398 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:36.950427 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:36.951918 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:36.951923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:36.953527 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:36.953548 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:36.954638 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:36.954646 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:36.954964 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:36.955967 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:02:36.956004 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:36.958865 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:36.962784 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:36.962798 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:36.962831 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:36.963440 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:36.963998 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:36.964004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:36.964008 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:36.964275 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:36.965218 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:36.965244 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:36.966043 (+ 799us) client_negotiation.cc:769] Sending connection context
0927 23:02:36.966525 (+ 482us) client_negotiation.cc:240] Negotiation successful
0927 23:02:36.967377 (+ 852us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":393,"threads_started":1}
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 23:02:36.971854 23022 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39545
I20260927 23:02:36.972343 23022 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:36.973452 23022 heartbeater.cc:511] Master 127.4.74.190:39545 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
I20260927 23:02:36.975809 22699 ts_manager.cc:194] Registered new tserver with Master: 89271be385e54c0e84987413f92977ef (127.4.74.130:40149)
I20260927 23:02:36.976835 22699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:45915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 23:02:37.013458 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39545
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:38167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:37.368257 23028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:37.368568 23028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:37.368695 23028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:37.376488 23028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:37.376623 23028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:37.376741 23028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:02:37.386919 23028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:39545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:37.388441 23028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:37.389833 23028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 23:02:37.395043 22888 heartbeater.cc:503] Master 127.4.74.190:39545 was elected leader, sending a full tablet report...
W20260927 23:02:37.407172 23034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:37.407418 23036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:37.407910 23033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:37.408980 23028 server_base.cc:1034] running on GCE node
I20260927 23:02:37.409895 23028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:37.411181 23028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:37.412413 23028 hybrid_clock.cc:648] HybridClock initialized: now 1790550157412322 us; error 82 us; skew 500 ppm
I20260927 23:02:37.412832 23028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:37 dist-test-slave-5pd3 krb5kdc[22654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550157, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:37.423867 23028 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:02:37.426018 23028 webserver.cc:460] Webserver started at http://127.4.74.131:41977/ using document root <none> and password file <none>
I20260927 23:02:37.426983 23028 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:37.427173 23028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:37.427712 23028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:37.432025 23028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "b890a949b807451887470f759fdf14ff"
format_stamp: "Formatted at 2026-09-27 23:02:37 on dist-test-slave-5pd3"
server_key: "a184d9630e62d6d3f3e023d36f4e505f"
server_key_iv: "6857969007db1ee82b400d7096dedd79"
server_key_version: "encryptionkey@0"
I20260927 23:02:37.433048 23028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "b890a949b807451887470f759fdf14ff"
format_stamp: "Formatted at 2026-09-27 23:02:37 on dist-test-slave-5pd3"
server_key: "a184d9630e62d6d3f3e023d36f4e505f"
server_key_iv: "6857969007db1ee82b400d7096dedd79"
server_key_version: "encryptionkey@0"
I20260927 23:02:37.440056 23028 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260927 23:02:37.444953 23043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:37.446856 23028 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260927 23:02:37.447114 23028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "b890a949b807451887470f759fdf14ff"
format_stamp: "Formatted at 2026-09-27 23:02:37 on dist-test-slave-5pd3"
server_key: "a184d9630e62d6d3f3e023d36f4e505f"
server_key_iv: "6857969007db1ee82b400d7096dedd79"
server_key_version: "encryptionkey@0"
I20260927 23:02:37.447396 23028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:37.479362 23028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:37.484938 23028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:37.485294 23028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:37.486522 23028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:37.488443 23028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:37.488559 23028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:37.488732 23028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:37.489070 23028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:37.532318 23028 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:43119
I20260927 23:02:37.532406 23155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:43119 every 8 connection(s)
I20260927 23:02:37.534066 23028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:02:37.543830 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 23028
I20260927 23:02:37.544056 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 23:02:37.544564 4394 external_mini_cluster.cc:1428] Setting key 8baef3492448fcf9d9ca09f945647a75
Sep 27 23:02:37 dist-test-slave-5pd3 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550157, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:37.561010 23158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.537530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:59149 (local address 127.4.74.190:39545)
0927 23:02:37.538181 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:37.538186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:37.539239 (+ 1053us) server_negotiation.cc:407] Connection header received
0927 23:02:37.541562 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:37.541568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:37.541699 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:37.541837 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:37.543960 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.544996 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.547761 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.548265 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.548420 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.553185 (+ 4765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:37.553193 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:37.553197 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:37.553252 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:37.556693 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:37.557786 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:37.557790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:37.557793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:37.557955 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:37.558505 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:37.558509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:37.558511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:37.558853 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:37.558999 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:37.560189 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0927 23:02:37.560537 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":368,"threads_started":1}
I20260927 23:02:37.562103 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.538008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:39545 (local address 127.4.74.131:59149)
0927 23:02:37.538973 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:37.539039 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:37.541208 (+ 2169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:37.542118 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:37.542128 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:37.542751 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:37.543591 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.543624 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.545272 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.545277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.547573 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.547593 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.548526 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.548535 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.548879 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.549905 (+ 1026us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:02:37.549931 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:37.552829 (+ 2898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:37.556947 (+ 4118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:37.556957 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:37.556973 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:37.557593 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:37.558097 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:37.558101 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:37.558105 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:37.558357 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:37.559005 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:37.559012 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:37.559460 (+ 448us) client_negotiation.cc:769] Sending connection context
0927 23:02:37.559938 (+ 478us) client_negotiation.cc:240] Negotiation successful
0927 23:02:37.560558 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":365,"threads_started":1}
I20260927 23:02:37.563946 23156 heartbeater.cc:347] Connected to a master server at 127.4.74.190:39545
I20260927 23:02:37.564278 23156 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:37.565161 23156 heartbeater.cc:511] Master 127.4.74.190:39545 requested a full tablet report, sending...
I20260927 23:02:37.567242 22699 ts_manager.cc:194] Registered new tserver with Master: b890a949b807451887470f759fdf14ff (127.4.74.131:43119)
I20260927 23:02:37.568393 22699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:59149
I20260927 23:02:37.575407 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:02:37.597277 23158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.580981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.4.74.190:39545)
0927 23:02:37.581256 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:37.581267 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:37.581787 (+ 520us) server_negotiation.cc:407] Connection header received
0927 23:02:37.582063 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:37.582068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:37.582165 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:37.582289 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:37.584161 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.584990 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.586488 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.586929 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.587083 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.590485 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:37.590491 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:37.590496 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:37.590543 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:37.593710 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:37.594747 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:37.594753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:37.594756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:37.594901 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:37.595387 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:37.595399 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:37.595402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:37.595868 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:37.596216 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:37.596642 (+ 426us) server_negotiation.cc:299] Negotiation successful
0927 23:02:37.596897 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260927 23:02:37.602559 22699 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:02:37.610621 22699 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:02:37.643006 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.628632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:43119 (local address 127.0.0.1:48514)
0927 23:02:37.631294 (+ 2662us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:37.631316 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:37.631493 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:37.632653 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:37.632658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:37.632696 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:37.633247 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.633264 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.635403 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.635408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.639036 (+ 3628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.639064 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.641601 (+ 2537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.641611 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.641845 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.641933 (+ 88us) client_negotiation.cc:769] Sending connection context
0927 23:02:37.642063 (+ 130us) client_negotiation.cc:240] Negotiation successful
0927 23:02:37.642393 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2520,"mutex_wait_us":3,"thread_start_us":705,"threads_started":1}
I20260927 23:02:37.643620 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.629459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48514 (local address 127.4.74.131:43119)
0927 23:02:37.630712 (+ 1253us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:37.630739 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:37.631625 (+ 886us) server_negotiation.cc:407] Connection header received
0927 23:02:37.631837 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:37.631846 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:37.632127 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:37.632342 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:37.633477 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.634957 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.639366 (+ 4409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.641298 (+ 1932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.642646 (+ 1348us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.642869 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:37.642945 (+ 76us) server_negotiation.cc:299] Negotiation successful
0927 23:02:37.643177 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1031,"thread_start_us":876,"threads_started":1}
I20260927 23:02:37.643770 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.631950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48570 (local address 127.4.74.129:41867)
0927 23:02:37.633213 (+ 1263us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:37.633234 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:37.634072 (+ 838us) server_negotiation.cc:407] Connection header received
0927 23:02:37.634209 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:37.634217 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:37.634518 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:37.634751 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:37.635785 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.637296 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.639880 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.641544 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.642269 (+ 725us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.642522 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:37.643133 (+ 611us) server_negotiation.cc:299] Negotiation successful
0927 23:02:37.643359 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":830,"threads_started":1}
I20260927 23:02:37.643777 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.633214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:41867 (local address 127.0.0.1:48570)
0927 23:02:37.633911 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:37.633935 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:37.634067 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:37.634774 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:37.634778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:37.634804 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:37.635207 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.635220 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.637470 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.637477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.639619 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.639641 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.642680 (+ 3039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.642687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.642875 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.642909 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 23:02:37.643036 (+ 127us) client_negotiation.cc:240] Negotiation successful
0927 23:02:37.643202 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":29,"thread_start_us":383,"threads_started":1}
I20260927 23:02:37.644420 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.627968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:40149 (local address 127.0.0.1:39398)
0927 23:02:37.630042 (+ 2074us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:37.630110 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:37.630414 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:37.632129 (+ 1715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:37.632139 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:37.632181 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:37.632931 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.632946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.635626 (+ 2680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.635633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.639122 (+ 3489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.639136 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.641791 (+ 2655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.641798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.642031 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.642079 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 23:02:37.642191 (+ 112us) client_negotiation.cc:240] Negotiation successful
0927 23:02:37.642377 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1598,"mutex_wait_us":43,"thread_start_us":3006,"threads_started":1}
I20260927 23:02:37.645632 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.628180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39398 (local address 127.4.74.130:40149)
0927 23:02:37.631235 (+ 3055us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:37.631247 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:37.631284 (+ 37us) server_negotiation.cc:407] Connection header received
0927 23:02:37.631431 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:37.631440 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:37.631828 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:37.633185 (+ 1357us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:37.633493 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.635433 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.639400 (+ 3967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.641139 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.641331 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.641561 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:37.644977 (+ 3416us) server_negotiation.cc:299] Negotiation successful
0927 23:02:37.645234 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2863,"thread_start_us":2652,"threads_started":1}
I20260927 23:02:37.649067 23091 tablet_service.cc:1467] Processing CreateTablet for tablet 3b8e9f3a22634349ae73d9540ffa2b3e (DEFAULT_TABLE table=test-table [id=093c77eae01e49c8aa7247bcf5d045da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:37.649096 22823 tablet_service.cc:1467] Processing CreateTablet for tablet 3b8e9f3a22634349ae73d9540ffa2b3e (DEFAULT_TABLE table=test-table [id=093c77eae01e49c8aa7247bcf5d045da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:37.650862 23091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b8e9f3a22634349ae73d9540ffa2b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:37.651345 22823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b8e9f3a22634349ae73d9540ffa2b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:37.650477 22957 tablet_service.cc:1467] Processing CreateTablet for tablet 3b8e9f3a22634349ae73d9540ffa2b3e (DEFAULT_TABLE table=test-table [id=093c77eae01e49c8aa7247bcf5d045da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:37.652724 22957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b8e9f3a22634349ae73d9540ffa2b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:37.666352 23175 tablet_bootstrap.cc:492] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: Bootstrap starting.
I20260927 23:02:37.670346 23175 tablet_bootstrap.cc:654] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:37.670971 23176 tablet_bootstrap.cc:492] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: Bootstrap starting.
I20260927 23:02:37.673768 23175 log.cc:824] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:37.675220 23176 tablet_bootstrap.cc:654] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:37.675756 23177 tablet_bootstrap.cc:492] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: Bootstrap starting.
I20260927 23:02:37.676851 23176 log.cc:824] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:37.678985 23175 tablet_bootstrap.cc:492] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: No bootstrap required, opened a new log
I20260927 23:02:37.679359 23175 ts_tablet_manager.cc:1397] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260927 23:02:37.679805 23177 tablet_bootstrap.cc:654] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:37.681100 23176 tablet_bootstrap.cc:492] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: No bootstrap required, opened a new log
I20260927 23:02:37.681164 23177 log.cc:824] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:37.681443 23176 ts_tablet_manager.cc:1397] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260927 23:02:37.685702 23177 tablet_bootstrap.cc:492] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: No bootstrap required, opened a new log
I20260927 23:02:37.686040 23177 ts_tablet_manager.cc:1397] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260927 23:02:37.689679 23175 raft_consensus.cc:348] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 } } peers { permanent_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41867 } } peers { permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 } }
I20260927 23:02:37.690227 23175 raft_consensus.cc:374] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:37.690356 23175 raft_consensus.cc:729] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b890a949b807451887470f759fdf14ff, State: Initialized, Role: FOLLOWER
I20260927 23:02:37.691182 23175 consensus_queue.cc:260] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 } } peers { permanent_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41867 } } peers { permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 } }
I20260927 23:02:37.692618 23176 raft_consensus.cc:348] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 } } peers { permanent_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41867 } } peers { permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 } }
I20260927 23:02:37.693172 23176 raft_consensus.cc:374] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:37.693382 23176 raft_consensus.cc:729] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08f44293698a4c2b8b7e49d1f28c70b0, State: Initialized, Role: FOLLOWER
I20260927 23:02:37.694176 23156 heartbeater.cc:503] Master 127.4.74.190:39545 was elected leader, sending a full tablet report...
I20260927 23:02:37.694823 23175 ts_tablet_manager.cc:1428] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: Time spent starting tablet: real 0.015s user 0.013s sys 0.004s
I20260927 23:02:37.695627 23176 consensus_queue.cc:260] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 } } peers { permanent_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41867 } } peers { permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 } }
I20260927 23:02:37.698107 23176 ts_tablet_manager.cc:1428] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
I20260927 23:02:37.699201 23177 raft_consensus.cc:348] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 } } peers { permanent_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41867 } } peers { permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 } }
I20260927 23:02:37.699975 23177 raft_consensus.cc:374] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:37.700124 23177 raft_consensus.cc:729] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89271be385e54c0e84987413f92977ef, State: Initialized, Role: FOLLOWER
I20260927 23:02:37.701017 23177 consensus_queue.cc:260] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 } } peers { permanent_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41867 } } peers { permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 } }
I20260927 23:02:37.702541 23022 heartbeater.cc:503] Master 127.4.74.190:39545 was elected leader, sending a full tablet report...
I20260927 23:02:37.703481 23177 ts_tablet_manager.cc:1428] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
W20260927 23:02:37.786386 23157 tablet.cc:2369] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:37.802037 23182 raft_consensus.cc:482] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:02:37.802634 23182 raft_consensus.cc:504] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 } } peers { permanent_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41867 } } peers { permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 } }
I20260927 23:02:37.805636 23182 leader_election.cc:288] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89271be385e54c0e84987413f92977ef (127.4.74.130:40149), b890a949b807451887470f759fdf14ff (127.4.74.131:43119)
I20260927 23:02:37.813800 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.806317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:33755 (local address 127.4.74.130:40149)
0927 23:02:37.806581 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:37.806590 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:37.807712 (+ 1122us) server_negotiation.cc:407] Connection header received
0927 23:02:37.807974 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:37.807979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:37.808047 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:37.808156 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:37.808902 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.809706 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.811893 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.812711 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.812936 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.813016 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:37.813323 (+ 307us) server_negotiation.cc:299] Negotiation successful
0927 23:02:37.813461 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260927 23:02:37.813969 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.806451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:40149 (local address 127.4.74.129:33755)
0927 23:02:37.807508 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:37.807527 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:37.807770 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:37.808184 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:37.808189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:37.808217 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:37.808710 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.808724 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.809872 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.809880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.811716 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.811730 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.812953 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.812959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.813138 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.813168 (+ 30us) client_negotiation.cc:769] Sending connection context
0927 23:02:37.813297 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 23:02:37.813461 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":1,"thread_start_us":670,"threads_started":1}
I20260927 23:02:37.814180 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.806526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:35213 (local address 127.4.74.131:43119)
0927 23:02:37.806847 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:37.806858 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:37.807610 (+ 752us) server_negotiation.cc:407] Connection header received
0927 23:02:37.807943 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:37.807948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:37.808024 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:37.808142 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:37.808935 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.809788 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.812017 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.812899 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.813150 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.813242 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:37.813561 (+ 319us) server_negotiation.cc:299] Negotiation successful
0927 23:02:37.813720 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260927 23:02:37.814378 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.806411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:43119 (local address 127.4.74.129:35213)
0927 23:02:37.807399 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:37.807429 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:37.807744 (+ 315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:37.808160 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:37.808165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:37.808191 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:37.808768 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.808779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.809953 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.809960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.811835 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:37.811851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.813178 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.813185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:37.813355 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.813397 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 23:02:37.813527 (+ 130us) client_negotiation.cc:240] Negotiation successful
0927 23:02:37.813718 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":59,"thread_start_us":444,"threads_started":1}
I20260927 23:02:37.815483 22977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8e9f3a22634349ae73d9540ffa2b3e" candidate_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89271be385e54c0e84987413f92977ef" is_pre_election: true
I20260927 23:02:37.815872 23111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8e9f3a22634349ae73d9540ffa2b3e" candidate_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b890a949b807451887470f759fdf14ff" is_pre_election: true
I20260927 23:02:37.816255 22977 raft_consensus.cc:2454] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08f44293698a4c2b8b7e49d1f28c70b0 in term 0.
I20260927 23:02:37.816500 23111 raft_consensus.cc:2454] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08f44293698a4c2b8b7e49d1f28c70b0 in term 0.
I20260927 23:02:37.817289 22777 leader_election.cc:302] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08f44293698a4c2b8b7e49d1f28c70b0, 89271be385e54c0e84987413f92977ef; no voters:
I20260927 23:02:37.817840 23182 raft_consensus.cc:2781] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:02:37.818009 23182 raft_consensus.cc:482] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:02:37.818146 23182 raft_consensus.cc:3037] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:37.821705 23182 raft_consensus.cc:504] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 } } peers { permanent_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41867 } } peers { permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 } }
I20260927 23:02:37.823438 23182 leader_election.cc:288] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [CANDIDATE]: Term 1 election: Requested vote from peers 89271be385e54c0e84987413f92977ef (127.4.74.130:40149), b890a949b807451887470f759fdf14ff (127.4.74.131:43119)
I20260927 23:02:37.823892 22977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8e9f3a22634349ae73d9540ffa2b3e" candidate_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89271be385e54c0e84987413f92977ef"
I20260927 23:02:37.824389 22977 raft_consensus.cc:3037] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:37.824395 23111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b8e9f3a22634349ae73d9540ffa2b3e" candidate_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b890a949b807451887470f759fdf14ff"
I20260927 23:02:37.824687 23111 raft_consensus.cc:3037] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:37.828773 22977 raft_consensus.cc:2454] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08f44293698a4c2b8b7e49d1f28c70b0 in term 1.
I20260927 23:02:37.828769 23111 raft_consensus.cc:2454] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08f44293698a4c2b8b7e49d1f28c70b0 in term 1.
I20260927 23:02:37.829509 22777 leader_election.cc:302] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 08f44293698a4c2b8b7e49d1f28c70b0, 89271be385e54c0e84987413f92977ef, b890a949b807451887470f759fdf14ff; no voters:
I20260927 23:02:37.830091 23182 raft_consensus.cc:2781] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:37.830879 23182 raft_consensus.cc:686] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 1 LEADER]: Becoming Leader. State: Replica: 08f44293698a4c2b8b7e49d1f28c70b0, State: Running, Role: LEADER
I20260927 23:02:37.831725 23182 consensus_queue.cc:237] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 } } peers { permanent_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41867 } } peers { permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 } }
I20260927 23:02:37.838171 22699 catalog_manager.cc:5495] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08f44293698a4c2b8b7e49d1f28c70b0 (127.4.74.129). New cstate: current_term: 1 leader_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08f44293698a4c2b8b7e49d1f28c70b0" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 41867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 } health_report { overall_health: UNKNOWN } } }
W20260927 23:02:37.857192 22889 tablet.cc:2369] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:37.894893 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.888086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48578 (local address 127.4.74.129:41867)
0927 23:02:37.888360 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:37.888366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:37.888397 (+ 31us) server_negotiation.cc:407] Connection header received
0927 23:02:37.888549 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:37.888554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:37.888681 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:37.888882 (+ 201us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:02:37.889370 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.890326 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.892216 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.892744 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.892887 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.893501 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:02:37.894219 (+ 718us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:02:37.894375 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:37.894575 (+ 200us) server_negotiation.cc:299] Negotiation successful
0927 23:02:37.894632 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260927 23:02:37.905869 22977 raft_consensus.cc:1261] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef [term 1 FOLLOWER]: Refusing update from remote peer 08f44293698a4c2b8b7e49d1f28c70b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:37.905874 23111 raft_consensus.cc:1261] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff [term 1 FOLLOWER]: Refusing update from remote peer 08f44293698a4c2b8b7e49d1f28c70b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:37.907456 23186 consensus_queue.cc:1035] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b890a949b807451887470f759fdf14ff" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 43119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:37.908120 23182 consensus_queue.cc:1035] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89271be385e54c0e84987413f92977ef" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 40149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:37.921139 23189 mvcc.cc:205] Tried to move back new op lower bound from 7334093446767349760 to 7334093446484422656. Current Snapshot: MvccSnapshot[applied={T|T < 7334093446767349760}]
I20260927 23:02:37.926062 23190 mvcc.cc:205] Tried to move back new op lower bound from 7334093446767349760 to 7334093446484422656. Current Snapshot: MvccSnapshot[applied={T|T < 7334093446767349760}]
I20260927 23:02:37.941402 22698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:37.942090 22698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:37.948907 22698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf9ecd1cb58847829f47f6c7a7585d6a: Sending DeleteTablet for 3 replicas of tablet 3b8e9f3a22634349ae73d9540ffa2b3e
I20260927 23:02:37.950802 23091 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8e9f3a22634349ae73d9540ffa2b3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:37 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:48514
I20260927 23:02:37.951900 22823 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8e9f3a22634349ae73d9540ffa2b3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:37 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:48570
I20260927 23:02:37.951454 22957 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b8e9f3a22634349ae73d9540ffa2b3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:37 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:39398
I20260927 23:02:37.961124 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 22760
I20260927 23:02:37.965992 23198 tablet_replica.cc:330] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: stopping tablet replica
I20260927 23:02:37.966755 23198 raft_consensus.cc:2229] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:37.967453 23198 raft_consensus.cc:2258] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:37.970806 23198 ts_tablet_manager.cc:1905] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:37.971900 23200 tablet_replica.cc:330] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: stopping tablet replica
I20260927 23:02:37.972673 23200 raft_consensus.cc:2229] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:37.973035 23199 tablet_replica.cc:330] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: stopping tablet replica
I20260927 23:02:37.973366 23200 raft_consensus.cc:2258] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:37.974079 23199 raft_consensus.cc:2229] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:02:37.975066 23199 raft_consensus.cc:2258] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:37.978129 23199 ts_tablet_manager.cc:1905] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:37.980213 23200 ts_tablet_manager.cc:1905] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:37.980924 23198 ts_tablet_manager.cc:1918] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:37.981144 23198 log.cc:1196] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/3b8e9f3a22634349ae73d9540ffa2b3e
I20260927 23:02:37.981791 23198 ts_tablet_manager.cc:1939] T 3b8e9f3a22634349ae73d9540ffa2b3e P b890a949b807451887470f759fdf14ff: Deleting consensus metadata
Sep 27 23:02:37 dist-test-slave-5pd3 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 23:02:37.984361 22686 catalog_manager.cc:4841] TS b890a949b807451887470f759fdf14ff (127.4.74.131:43119): tablet 3b8e9f3a22634349ae73d9540ffa2b3e (table test-table [id=093c77eae01e49c8aa7247bcf5d045da]) successfully deleted
I20260927 23:02:37.991885 23200 ts_tablet_manager.cc:1918] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:37.992292 23200 log.cc:1196] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/3b8e9f3a22634349ae73d9540ffa2b3e
I20260927 23:02:37.993072 23200 ts_tablet_manager.cc:1939] T 3b8e9f3a22634349ae73d9540ffa2b3e P 89271be385e54c0e84987413f92977ef: Deleting consensus metadata
I20260927 23:02:37.994376 23199 ts_tablet_manager.cc:1918] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:37.994611 23199 log.cc:1196] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/3b8e9f3a22634349ae73d9540ffa2b3e
I20260927 23:02:37.995221 23199 ts_tablet_manager.cc:1939] T 3b8e9f3a22634349ae73d9540ffa2b3e P 08f44293698a4c2b8b7e49d1f28c70b0: Deleting consensus metadata
I20260927 23:02:37.995905 22684 catalog_manager.cc:4841] TS 89271be385e54c0e84987413f92977ef (127.4.74.130:40149): tablet 3b8e9f3a22634349ae73d9540ffa2b3e (table test-table [id=093c77eae01e49c8aa7247bcf5d045da]) successfully deleted
I20260927 23:02:37.997007 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:37.963233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.4.74.129:41867)
0927 23:02:37.963497 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:37.963504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:37.969098 (+ 5594us) server_negotiation.cc:407] Connection header received
0927 23:02:37.969483 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:37.969492 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:37.969608 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:37.969744 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:37.973838 (+ 4094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.975146 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.977839 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:37.978595 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:37.978718 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:37.985112 (+ 6394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:37.985136 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:37.985161 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:37.985230 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:37.990945 (+ 5715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:37.992392 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:37.992405 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:37.992409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:37.992688 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:37.993248 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:37.993255 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:37.993261 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:37.993866 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:37.994196 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:37.996412 (+ 2216us) server_negotiation.cc:299] Negotiation successful
0927 23:02:37.996683 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260927 23:02:37.997651 22684 catalog_manager.cc:4841] TS 08f44293698a4c2b8b7e49d1f28c70b0 (127.4.74.129:41867): tablet 3b8e9f3a22634349ae73d9540ffa2b3e (table test-table [id=093c77eae01e49c8aa7247bcf5d045da]) successfully deleted
I20260927 23:02:37.997982 22883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:38.111014 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 22760
I20260927 23:02:38.134814 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 22894
Sep 27 23:02:38 dist-test-slave-5pd3 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 23:02:38.155481 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:38.136121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.4.74.130:40149)
0927 23:02:38.136418 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:38.136424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:38.136450 (+ 26us) server_negotiation.cc:407] Connection header received
0927 23:02:38.136696 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:38.136702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:38.136841 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:38.136964 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:38.138522 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:38.139321 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:38.140973 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:38.141411 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:38.141539 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:38.147526 (+ 5987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:38.147623 (+ 97us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:38.147650 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:38.147724 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:38.150869 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:38.152602 (+ 1733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:38.152619 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:38.152622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:38.152784 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:38.153595 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:38.153600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:38.153602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:38.154153 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:38.154415 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:38.154895 (+ 480us) server_negotiation.cc:299] Negotiation successful
0927 23:02:38.155103 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260927 23:02:38.156570 23017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:38.270010 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 22894
I20260927 23:02:38.289538 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 23028
Sep 27 23:02:38 dist-test-slave-5pd3 krb5kdc[22654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 23:02:38.310213 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:38.290635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48530 (local address 127.4.74.131:43119)
0927 23:02:38.290922 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:38.290931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:38.291011 (+ 80us) server_negotiation.cc:407] Connection header received
0927 23:02:38.291256 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:38.291262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:38.291364 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:38.291485 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:38.293043 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:38.293798 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:38.295165 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:38.295701 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:38.295873 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:38.301660 (+ 5787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:38.301725 (+ 65us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:38.301741 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:38.301788 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:38.305696 (+ 3908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:38.307082 (+ 1386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:38.307091 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:38.307093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:38.307371 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:38.308172 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:38.308177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:38.308179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:38.308665 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:38.308953 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:38.309481 (+ 528us) server_negotiation.cc:299] Negotiation successful
0927 23:02:38.309745 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260927 23:02:38.311254 23151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:38.408486 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 23028
I20260927 23:02:38.428411 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 22667
I20260927 23:02:38.429863 22729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:38.569890 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 22667
2026-09-27T23:02:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4897 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-27T23:02:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:02:38Z Disabled control of system clock
I20260927 23:02:38.659125 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36325
--webserver_interface=127.4.74.190
--webserver_port=0
--builtin_ntp_servers=127.4.74.148:35263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 23:02:39.038350 23209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:39.038700 23209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:39.038800 23209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:39.046736 23209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:39.046883 23209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:39.046922 23209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:39.046974 23209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:39.047014 23209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:39.057240 23209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36325
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:39.058712 23209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:39.060711 23209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:39.075929 23214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:39.076104 23217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:39.076017 23215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:39.077929 23209 server_base.cc:1034] running on GCE node
I20260927 23:02:39.078920 23209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:39.080710 23209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:39.082020 23209 hybrid_clock.cc:648] HybridClock initialized: now 1790550159081961 us; error 61 us; skew 500 ppm
I20260927 23:02:39.082464 23209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:39.085407 23209 webserver.cc:460] Webserver started at http://127.4.74.190:38851/ using document root <none> and password file <none>
I20260927 23:02:39.086167 23209 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:39.086268 23209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:39.086553 23209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:39.089872 23209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "8bc8c7e625014f43b02e765d8f6bce6d"
format_stamp: "Formatted at 2026-09-27 23:02:39 on dist-test-slave-5pd3"
server_key: "8b7fc0ee6fe0d5a2b201aa79408e45bc"
server_key_iv: "7bd09a2bae5e64516d99a08b17c1507c"
server_key_version: "encryptionkey@0"
I20260927 23:02:39.090694 23209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "8bc8c7e625014f43b02e765d8f6bce6d"
format_stamp: "Formatted at 2026-09-27 23:02:39 on dist-test-slave-5pd3"
server_key: "8b7fc0ee6fe0d5a2b201aa79408e45bc"
server_key_iv: "7bd09a2bae5e64516d99a08b17c1507c"
server_key_version: "encryptionkey@0"
I20260927 23:02:39.097064 23209 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260927 23:02:39.101959 23223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:39.104211 23209 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260927 23:02:39.104437 23209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "8bc8c7e625014f43b02e765d8f6bce6d"
format_stamp: "Formatted at 2026-09-27 23:02:39 on dist-test-slave-5pd3"
server_key: "8b7fc0ee6fe0d5a2b201aa79408e45bc"
server_key_iv: "7bd09a2bae5e64516d99a08b17c1507c"
server_key_version: "encryptionkey@0"
I20260927 23:02:39.104664 23209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:39.126930 23209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:39.127975 23209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:39.128247 23209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:39.151639 23209 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:36325
I20260927 23:02:39.151818 23274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:36325 every 8 connection(s)
I20260927 23:02:39.153189 23209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:39.158378 23275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:39.160393 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 23209
I20260927 23:02:39.160622 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:02:39.161269 4394 external_mini_cluster.cc:1428] Setting key a155eac445caff88982b80536aa46f96
I20260927 23:02:39.175086 23275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d: Bootstrap starting.
I20260927 23:02:39.177466 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:39.163767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.4.74.190:36325)
0927 23:02:39.165113 (+ 1346us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:39.165131 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:39.165178 (+ 47us) server_negotiation.cc:407] Connection header received
0927 23:02:39.167078 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:39.167104 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:39.167785 (+ 681us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:39.168209 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:39.168839 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:39.170189 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:39.172178 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:39.173026 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:39.173261 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:39.174709 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:39.174718 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:39.174736 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:39.175413 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:39.175506 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:39.175536 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:39.175737 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:39.175980 (+ 243us) server_negotiation.cc:299] Negotiation successful
0927 23:02:39.176371 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":482,"threads_started":1}
I20260927 23:02:39.180182 23275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:39.181697 23275 log.cc:824] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:39.185295 23275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d: No bootstrap required, opened a new log
I20260927 23:02:39.194686 23275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc8c7e625014f43b02e765d8f6bce6d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36325 } }
I20260927 23:02:39.195317 23275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:39.195444 23275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bc8c7e625014f43b02e765d8f6bce6d, State: Initialized, Role: FOLLOWER
I20260927 23:02:39.196439 23275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc8c7e625014f43b02e765d8f6bce6d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36325 } }
I20260927 23:02:39.196820 23275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:39.197018 23275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:39.197268 23275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:39.200047 23275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc8c7e625014f43b02e765d8f6bce6d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36325 } }
I20260927 23:02:39.200803 23275 leader_election.cc:302] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bc8c7e625014f43b02e765d8f6bce6d; no voters:
I20260927 23:02:39.201632 23275 leader_election.cc:288] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:02:39.201923 23280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:39.202911 23280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [term 1 LEADER]: Becoming Leader. State: Replica: 8bc8c7e625014f43b02e765d8f6bce6d, State: Running, Role: LEADER
I20260927 23:02:39.203706 23280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc8c7e625014f43b02e765d8f6bce6d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36325 } }
I20260927 23:02:39.204813 23275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:39.207815 23282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bc8c7e625014f43b02e765d8f6bce6d. Latest consensus state: current_term: 1 leader_uuid: "8bc8c7e625014f43b02e765d8f6bce6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc8c7e625014f43b02e765d8f6bce6d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36325 } } }
I20260927 23:02:39.208652 23282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:39.208355 23281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bc8c7e625014f43b02e765d8f6bce6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc8c7e625014f43b02e765d8f6bce6d" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36325 } } }
I20260927 23:02:39.209362 23281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:39.210685 23287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:39.216784 23287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:39.226858 23287 catalog_manager.cc:1347] Generated new cluster ID: 94fc670baae545cfa2a7ecd013f8ea07
I20260927 23:02:39.227277 23287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:39.240074 23287 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:02:39.241499 23287 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:39.249063 23287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d: Generated new TSK 0
I20260927 23:02:39.250113 23287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:02:39.255414 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36325
--builtin_ntp_servers=127.4.74.148:35263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 23:02:39.609757 23299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:39.610147 23299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:39.610293 23299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:39.618464 23299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:39.618594 23299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:39.618770 23299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:02:39.628887 23299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:39.630362 23299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:39.632513 23299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:39.649772 23307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:39.649642 23304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:39.650190 23305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:39.651356 23299 server_base.cc:1034] running on GCE node
I20260927 23:02:39.652227 23299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:39.653529 23299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:39.654950 23299 hybrid_clock.cc:648] HybridClock initialized: now 1790550159654889 us; error 84 us; skew 500 ppm
I20260927 23:02:39.655354 23299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:39.658576 23299 webserver.cc:460] Webserver started at http://127.4.74.129:34651/ using document root <none> and password file <none>
I20260927 23:02:39.659436 23299 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:39.659618 23299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:39.660003 23299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:39.663278 23299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "9528fc3c3e9e4c95bd406813fd325685"
format_stamp: "Formatted at 2026-09-27 23:02:39 on dist-test-slave-5pd3"
server_key: "e993c8f8f3a8e613028bb06916193f1d"
server_key_iv: "d30e7668ae449ccbb0adae8180139661"
server_key_version: "encryptionkey@0"
I20260927 23:02:39.664261 23299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "9528fc3c3e9e4c95bd406813fd325685"
format_stamp: "Formatted at 2026-09-27 23:02:39 on dist-test-slave-5pd3"
server_key: "e993c8f8f3a8e613028bb06916193f1d"
server_key_iv: "d30e7668ae449ccbb0adae8180139661"
server_key_version: "encryptionkey@0"
I20260927 23:02:39.670693 23299 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:02:39.676059 23313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:39.678107 23299 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260927 23:02:39.678380 23299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "9528fc3c3e9e4c95bd406813fd325685"
format_stamp: "Formatted at 2026-09-27 23:02:39 on dist-test-slave-5pd3"
server_key: "e993c8f8f3a8e613028bb06916193f1d"
server_key_iv: "d30e7668ae449ccbb0adae8180139661"
server_key_version: "encryptionkey@0"
I20260927 23:02:39.678658 23299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:39.700526 23299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:39.701471 23299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:39.701831 23299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:39.703047 23299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:39.705044 23299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:39.705181 23299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:39.705353 23299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:39.705451 23299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:39.748718 23299 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:44623
I20260927 23:02:39.748852 23425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:44623 every 8 connection(s)
I20260927 23:02:39.750375 23299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:02:39.754657 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 23299
I20260927 23:02:39.754876 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 23:02:39.755540 4394 external_mini_cluster.cc:1428] Setting key c3b9e2d2d982cc3928a19a433c331537
I20260927 23:02:39.758949 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36325
--builtin_ntp_servers=127.4.74.148:35263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 23:02:39.768558 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:39.754179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:41773 (local address 127.4.74.190:36325)
0927 23:02:39.754862 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:39.754871 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:39.756646 (+ 1775us) server_negotiation.cc:407] Connection header received
0927 23:02:39.759239 (+ 2593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:39.759249 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:39.759372 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:39.759535 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:39.761141 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:39.762345 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:39.764526 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:39.765331 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:39.765521 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:39.767383 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:39.767395 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:39.767398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:39.767426 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:39.767482 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:39.767497 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:39.767675 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:39.767980 (+ 305us) server_negotiation.cc:299] Negotiation successful
0927 23:02:39.768131 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":356,"threads_started":1}
I20260927 23:02:39.770071 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:39.754838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36325 (local address 127.4.74.129:41773)
0927 23:02:39.756426 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:39.756493 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:39.758770 (+ 2277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:39.759884 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:39.759895 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:39.760030 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:39.760881 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:39.760904 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:39.762585 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:39.762591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:39.764287 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:39.764326 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:39.765623 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:39.765628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:39.765911 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:39.766852 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 23:02:39.766855 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:39.766913 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 23:02:39.767035 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:39.767668 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:39.767682 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:39.767694 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 23:02:39.767950 (+ 256us) client_negotiation.cc:240] Negotiation successful
0927 23:02:39.768555 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":441,"threads_started":1}
I20260927 23:02:39.772387 23426 heartbeater.cc:347] Connected to a master server at 127.4.74.190:36325
I20260927 23:02:39.772869 23426 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:39.773947 23426 heartbeater.cc:511] Master 127.4.74.190:36325 requested a full tablet report, sending...
I20260927 23:02:39.776820 23240 ts_manager.cc:194] Registered new tserver with Master: 9528fc3c3e9e4c95bd406813fd325685 (127.4.74.129:44623)
I20260927 23:02:39.778679 23240 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.129:41773
W20260927 23:02:40.173441 23430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:40.173830 23430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:40.173996 23430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:40.184461 23430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:40.184630 23430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:40.184808 23430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:02:40.195860 23430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:40.197360 23430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:40.199291 23430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:40.216959 23435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:40.217104 23436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:40.218026 23438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:40.218725 23430 server_base.cc:1034] running on GCE node
I20260927 23:02:40.219734 23430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:40.221092 23430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:40.222419 23430 hybrid_clock.cc:648] HybridClock initialized: now 1790550160222363 us; error 64 us; skew 500 ppm
I20260927 23:02:40.222853 23430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:40.226213 23430 webserver.cc:460] Webserver started at http://127.4.74.130:38571/ using document root <none> and password file <none>
I20260927 23:02:40.227111 23430 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:40.227216 23430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:40.227695 23430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:40.231383 23430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "961991ac868847399fb630a0d497b934"
format_stamp: "Formatted at 2026-09-27 23:02:40 on dist-test-slave-5pd3"
server_key: "395ccc109fbae29252e00f26edb79804"
server_key_iv: "468b46bf62a4d70fb45dd54abf265658"
server_key_version: "encryptionkey@0"
I20260927 23:02:40.232434 23430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "961991ac868847399fb630a0d497b934"
format_stamp: "Formatted at 2026-09-27 23:02:40 on dist-test-slave-5pd3"
server_key: "395ccc109fbae29252e00f26edb79804"
server_key_iv: "468b46bf62a4d70fb45dd54abf265658"
server_key_version: "encryptionkey@0"
I20260927 23:02:40.238982 23430 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:02:40.244055 23444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:40.246069 23430 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:02:40.246377 23430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "961991ac868847399fb630a0d497b934"
format_stamp: "Formatted at 2026-09-27 23:02:40 on dist-test-slave-5pd3"
server_key: "395ccc109fbae29252e00f26edb79804"
server_key_iv: "468b46bf62a4d70fb45dd54abf265658"
server_key_version: "encryptionkey@0"
I20260927 23:02:40.246687 23430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:40.270295 23430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:40.271299 23430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:40.271728 23430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:40.272944 23430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:40.274780 23430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:40.274880 23430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:40.275056 23430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:40.275151 23430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:40.316299 23430 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:43425
I20260927 23:02:40.316416 23556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:43425 every 8 connection(s)
I20260927 23:02:40.317803 23430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:02:40.323695 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 23430
I20260927 23:02:40.323920 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 23:02:40.325165 4394 external_mini_cluster.cc:1428] Setting key 1376e63ab590c8b878ca250cc79db22e
I20260927 23:02:40.329478 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36325
--builtin_ntp_servers=127.4.74.148:35263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 23:02:40.337453 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.322408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:41717 (local address 127.4.74.190:36325)
0927 23:02:40.323111 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:40.323116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:40.325077 (+ 1961us) server_negotiation.cc:407] Connection header received
0927 23:02:40.328593 (+ 3516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:40.328599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:40.328729 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:40.328874 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:40.330654 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.331517 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.333670 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.334158 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.334333 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.336337 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:40.336344 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:40.336347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:40.336377 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:40.336430 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:40.336445 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:40.336558 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:40.336873 (+ 315us) server_negotiation.cc:299] Negotiation successful
0927 23:02:40.337015 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":397,"threads_started":1}
I20260927 23:02:40.339130 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.322971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36325 (local address 127.4.74.130:41717)
0927 23:02:40.324785 (+ 1814us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:40.324848 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:40.328089 (+ 3241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:40.329239 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:40.329251 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:40.329397 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:40.330402 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:40.330427 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.331815 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.331820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:40.333444 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:40.333464 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.334465 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.334473 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:40.334803 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.335839 (+ 1036us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 23:02:40.335842 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:40.335911 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 23:02:40.336040 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:40.336558 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:40.336573 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:40.336586 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 23:02:40.336851 (+ 265us) client_negotiation.cc:240] Negotiation successful
0927 23:02:40.337458 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":564,"threads_started":1}
I20260927 23:02:40.341715 23557 heartbeater.cc:347] Connected to a master server at 127.4.74.190:36325
I20260927 23:02:40.342257 23557 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:40.343328 23557 heartbeater.cc:511] Master 127.4.74.190:36325 requested a full tablet report, sending...
I20260927 23:02:40.345722 23240 ts_manager.cc:194] Registered new tserver with Master: 961991ac868847399fb630a0d497b934 (127.4.74.130:43425)
I20260927 23:02:40.346783 23240 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.130:41717
W20260927 23:02:40.689560 23561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:40.689954 23561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:40.690122 23561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:40.697854 23561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:40.698037 23561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:40.698211 23561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:02:40.708398 23561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:36325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:40.710211 23561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:40.712376 23561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:40.729931 23566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:40.730020 23567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:40.730020 23569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:40.731482 23561 server_base.cc:1034] running on GCE node
I20260927 23:02:40.732352 23561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:40.733618 23561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:40.734910 23561 hybrid_clock.cc:648] HybridClock initialized: now 1790550160734848 us; error 61 us; skew 500 ppm
I20260927 23:02:40.735322 23561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:40.738312 23561 webserver.cc:460] Webserver started at http://127.4.74.131:34621/ using document root <none> and password file <none>
I20260927 23:02:40.739149 23561 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:40.739313 23561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:40.739782 23561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:40.743196 23561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "7669f278389b434ebb1fdfdd1cbe69e2"
format_stamp: "Formatted at 2026-09-27 23:02:40 on dist-test-slave-5pd3"
server_key: "34978a9bc3155d651cd57412fba13478"
server_key_iv: "f2051e2cf44d49bf808e345bc783719f"
server_key_version: "encryptionkey@0"
I20260927 23:02:40.744197 23561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "7669f278389b434ebb1fdfdd1cbe69e2"
format_stamp: "Formatted at 2026-09-27 23:02:40 on dist-test-slave-5pd3"
server_key: "34978a9bc3155d651cd57412fba13478"
server_key_iv: "f2051e2cf44d49bf808e345bc783719f"
server_key_version: "encryptionkey@0"
I20260927 23:02:40.751143 23561 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:02:40.756595 23575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:40.758544 23561 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:02:40.758845 23561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "7669f278389b434ebb1fdfdd1cbe69e2"
format_stamp: "Formatted at 2026-09-27 23:02:40 on dist-test-slave-5pd3"
server_key: "34978a9bc3155d651cd57412fba13478"
server_key_iv: "f2051e2cf44d49bf808e345bc783719f"
server_key_version: "encryptionkey@0"
I20260927 23:02:40.759164 23561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:40.782249 23561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:40.783321 23561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:40.783413 23426 heartbeater.cc:503] Master 127.4.74.190:36325 was elected leader, sending a full tablet report...
I20260927 23:02:40.783833 23561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:40.785172 23561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:40.787194 23561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:40.787297 23561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:40.787377 23561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:40.787446 23561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:40.830753 23561 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:38227
I20260927 23:02:40.830822 23687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:38227 every 8 connection(s)
I20260927 23:02:40.832677 23561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:02:40.841295 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 23561
I20260927 23:02:40.841538 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 23:02:40.842273 4394 external_mini_cluster.cc:1428] Setting key 1ebda0b1e93f774f36ff5e38d18b1e52
I20260927 23:02:40.859951 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.836797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:39795 (local address 127.4.74.190:36325)
0927 23:02:40.837142 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:40.837149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:40.839628 (+ 2479us) server_negotiation.cc:407] Connection header received
0927 23:02:40.841793 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:40.841801 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:40.841906 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:40.842047 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:40.843444 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.844662 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.846845 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.847336 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.847464 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.858766 (+ 11302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:40.858774 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:40.858780 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:40.858810 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:40.858873 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:40.858894 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:40.859033 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:40.859388 (+ 355us) server_negotiation.cc:299] Negotiation successful
0927 23:02:40.859612 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260927 23:02:40.861274 23690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.837799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36325 (local address 127.4.74.131:39795)
0927 23:02:40.839397 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:40.839443 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:40.841344 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:40.842319 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:40.842328 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:40.842488 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:40.843159 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:40.843187 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.844919 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.844930 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:40.846657 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:40.846674 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.847629 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.847640 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:40.847907 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.858085 (+ 10178us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 23:02:40.858088 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:40.858144 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 23:02:40.858443 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:40.859071 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:40.859083 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:40.859093 (+ 10us) client_negotiation.cc:769] Sending connection context
0927 23:02:40.859334 (+ 241us) client_negotiation.cc:240] Negotiation successful
0927 23:02:40.859849 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":616,"threads_started":1}
I20260927 23:02:40.863369 23688 heartbeater.cc:347] Connected to a master server at 127.4.74.190:36325
I20260927 23:02:40.863934 23688 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:40.864894 23688 heartbeater.cc:511] Master 127.4.74.190:36325 requested a full tablet report, sending...
I20260927 23:02:40.866962 23240 ts_manager.cc:194] Registered new tserver with Master: 7669f278389b434ebb1fdfdd1cbe69e2 (127.4.74.131:38227)
I20260927 23:02:40.867933 23240 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.131:39795
I20260927 23:02:40.874564 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:02:40.888921 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.880137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.4.74.190:36325)
0927 23:02:40.880402 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:40.880407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:40.881281 (+ 874us) server_negotiation.cc:407] Connection header received
0927 23:02:40.881447 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:40.881452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:40.881537 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:40.881668 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:40.882261 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.883312 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.885384 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.885894 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.886068 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.887939 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:40.887945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:40.887949 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:40.887984 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:40.888030 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:40.888044 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:40.888229 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:40.888450 (+ 221us) server_negotiation.cc:299] Negotiation successful
0927 23:02:40.888529 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260927 23:02:40.894518 23240 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:02:40.902655 23240 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:02:40.943112 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.921397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.4.74.130:43425)
0927 23:02:40.929462 (+ 8065us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:40.929480 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:40.932041 (+ 2561us) server_negotiation.cc:407] Connection header received
0927 23:02:40.932176 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:40.932187 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:40.932680 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:40.934242 (+ 1562us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:40.934455 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.936239 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.940455 (+ 4216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.941738 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.941908 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.942204 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:40.942518 (+ 314us) server_negotiation.cc:299] Negotiation successful
0927 23:02:40.942714 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7014,"thread_start_us":6787,"threads_started":1}
I20260927 23:02:40.943642 23704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.921193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:43425 (local address 127.0.0.1:55310)
0927 23:02:40.930698 (+ 9505us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:40.930783 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:40.931156 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:40.933239 (+ 2083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:40.933247 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:40.933283 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:40.933955 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:40.933971 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.936448 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.936455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:40.940075 (+ 3620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:40.940103 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.941976 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.941983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:40.942193 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.942262 (+ 69us) client_negotiation.cc:769] Sending connection context
0927 23:02:40.942787 (+ 525us) client_negotiation.cc:240] Negotiation successful
0927 23:02:40.943009 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8979,"mutex_wait_us":172,"thread_start_us":10432,"threads_started":1}
I20260927 23:02:40.945331 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.922013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44623 (local address 127.0.0.1:41568)
0927 23:02:40.931973 (+ 9960us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:40.931997 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:40.932127 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:40.933659 (+ 1532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:40.933666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:40.933704 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:40.934391 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:40.934409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.936448 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.936455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:40.940787 (+ 4332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:40.940801 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.944387 (+ 3586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.944393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:40.944655 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.944692 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 23:02:40.944797 (+ 105us) client_negotiation.cc:240] Negotiation successful
0927 23:02:40.944950 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9786,"mutex_wait_us":228,"thread_start_us":11785,"threads_started":1}
I20260927 23:02:40.946154 23703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.922426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41568 (local address 127.4.74.129:44623)
0927 23:02:40.933007 (+ 10581us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:40.933016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:40.933045 (+ 29us) server_negotiation.cc:407] Connection header received
0927 23:02:40.933175 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:40.933182 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:40.933419 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:40.933945 (+ 526us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:40.934627 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.936231 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.941095 (+ 4864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.943162 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.943890 (+ 728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.944299 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:40.945657 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0927 23:02:40.945850 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10348,"thread_start_us":10419,"threads_started":1}
I20260927 23:02:40.948352 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.922714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:38227 (local address 127.0.0.1:36396)
0927 23:02:40.934822 (+ 12108us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:40.934841 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:40.935063 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:40.935884 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:40.935889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:40.935918 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:40.936831 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:40.936856 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.939625 (+ 2769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.939631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:40.941769 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:40.941793 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.947046 (+ 5253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.947053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:40.947453 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.947509 (+ 56us) client_negotiation.cc:769] Sending connection context
0927 23:02:40.947709 (+ 200us) client_negotiation.cc:240] Negotiation successful
0927 23:02:40.947901 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11982,"mutex_wait_us":3,"thread_start_us":7291,"threads_started":1}
I20260927 23:02:40.948549 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:40.923711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.4.74.131:38227)
0927 23:02:40.929534 (+ 5823us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:40.929549 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:40.935050 (+ 5501us) server_negotiation.cc:407] Connection header received
0927 23:02:40.935250 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:40.935258 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:40.935651 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:40.935902 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:40.937220 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.939379 (+ 2159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.942115 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:40.946028 (+ 3913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:40.946184 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:40.946569 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:40.947836 (+ 1267us) server_negotiation.cc:299] Negotiation successful
0927 23:02:40.948088 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5496,"thread_start_us":5113,"threads_started":1}
I20260927 23:02:40.950519 23361 tablet_service.cc:1467] Processing CreateTablet for tablet 129126bac13946f1810454f153dac621 (DEFAULT_TABLE table=test-table [id=9101e929198b419ab1801f6293491373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:40.951290 23492 tablet_service.cc:1467] Processing CreateTablet for tablet 129126bac13946f1810454f153dac621 (DEFAULT_TABLE table=test-table [id=9101e929198b419ab1801f6293491373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:40.952330 23361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129126bac13946f1810454f153dac621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:40.954541 23492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129126bac13946f1810454f153dac621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:40.955829 23623 tablet_service.cc:1467] Processing CreateTablet for tablet 129126bac13946f1810454f153dac621 (DEFAULT_TABLE table=test-table [id=9101e929198b419ab1801f6293491373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:40.958664 23623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 129126bac13946f1810454f153dac621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:40.967375 23706 tablet_bootstrap.cc:492] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685: Bootstrap starting.
I20260927 23:02:40.970854 23706 tablet_bootstrap.cc:654] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:40.972123 23706 log.cc:824] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:40.979592 23708 tablet_bootstrap.cc:492] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: Bootstrap starting.
I20260927 23:02:40.979840 23709 tablet_bootstrap.cc:492] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: Bootstrap starting.
I20260927 23:02:40.983846 23709 tablet_bootstrap.cc:654] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:40.984720 23706 tablet_bootstrap.cc:492] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685: No bootstrap required, opened a new log
I20260927 23:02:40.984731 23708 tablet_bootstrap.cc:654] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:40.985157 23706 ts_tablet_manager.cc:1397] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.006s
I20260927 23:02:40.985157 23709 log.cc:824] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:40.986284 23708 log.cc:824] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:40.988646 23709 tablet_bootstrap.cc:492] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: No bootstrap required, opened a new log
I20260927 23:02:40.989006 23709 ts_tablet_manager.cc:1397] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260927 23:02:40.992631 23708 tablet_bootstrap.cc:492] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: No bootstrap required, opened a new log
I20260927 23:02:40.993095 23708 ts_tablet_manager.cc:1397] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260927 23:02:41.000981 23706 raft_consensus.cc:348] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961991ac868847399fb630a0d497b934" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43425 } } peers { permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 } } peers { permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 } }
I20260927 23:02:41.001864 23709 raft_consensus.cc:348] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961991ac868847399fb630a0d497b934" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43425 } } peers { permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 } } peers { permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 } }
I20260927 23:02:41.002274 23706 raft_consensus.cc:374] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:41.002411 23709 raft_consensus.cc:374] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:41.002444 23706 raft_consensus.cc:729] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9528fc3c3e9e4c95bd406813fd325685, State: Initialized, Role: FOLLOWER
I20260927 23:02:41.002524 23709 raft_consensus.cc:729] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 961991ac868847399fb630a0d497b934, State: Initialized, Role: FOLLOWER
I20260927 23:02:41.003401 23709 consensus_queue.cc:260] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961991ac868847399fb630a0d497b934" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43425 } } peers { permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 } } peers { permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 } }
I20260927 23:02:41.003845 23706 consensus_queue.cc:260] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961991ac868847399fb630a0d497b934" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43425 } } peers { permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 } } peers { permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 } }
I20260927 23:02:41.007043 23709 ts_tablet_manager.cc:1428] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260927 23:02:41.007743 23557 heartbeater.cc:503] Master 127.4.74.190:36325 was elected leader, sending a full tablet report...
I20260927 23:02:41.008741 23706 ts_tablet_manager.cc:1428] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685: Time spent starting tablet: real 0.023s user 0.021s sys 0.003s
I20260927 23:02:41.008826 23708 raft_consensus.cc:348] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961991ac868847399fb630a0d497b934" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43425 } } peers { permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 } } peers { permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 } }
I20260927 23:02:41.009611 23708 raft_consensus.cc:374] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:41.009805 23708 raft_consensus.cc:729] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7669f278389b434ebb1fdfdd1cbe69e2, State: Initialized, Role: FOLLOWER
I20260927 23:02:41.010643 23708 consensus_queue.cc:260] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961991ac868847399fb630a0d497b934" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43425 } } peers { permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 } } peers { permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 } }
I20260927 23:02:41.013063 23688 heartbeater.cc:503] Master 127.4.74.190:36325 was elected leader, sending a full tablet report...
I20260927 23:02:41.014130 23708 ts_tablet_manager.cc:1428] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: Time spent starting tablet: real 0.021s user 0.015s sys 0.002s
W20260927 23:02:41.070664 23558 tablet.cc:2369] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:41.075871 23712 raft_consensus.cc:482] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:02:41.076236 23712 raft_consensus.cc:504] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961991ac868847399fb630a0d497b934" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43425 } } peers { permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 } } peers { permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 } }
I20260927 23:02:41.078364 23712 leader_election.cc:288] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7669f278389b434ebb1fdfdd1cbe69e2 (127.4.74.131:38227), 9528fc3c3e9e4c95bd406813fd325685 (127.4.74.129:44623)
W20260927 23:02:41.085059 23689 tablet.cc:2369] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:41.089121 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.078801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:47097 (local address 127.4.74.131:38227)
0927 23:02:41.079080 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:41.079089 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:41.080877 (+ 1788us) server_negotiation.cc:407] Connection header received
0927 23:02:41.081224 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:41.081231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:41.081345 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:41.081491 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:41.082420 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.083388 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.086770 (+ 3382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.087795 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.088028 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:41.088087 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:41.088639 (+ 552us) server_negotiation.cc:299] Negotiation successful
0927 23:02:41.088810 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260927 23:02:41.089774 23703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.079208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:45993 (local address 127.4.74.129:44623)
0927 23:02:41.079439 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:41.079445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:41.080768 (+ 1323us) server_negotiation.cc:407] Connection header received
0927 23:02:41.080938 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:41.080944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:41.081072 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:41.081177 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:41.082058 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.083029 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.086985 (+ 3956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.088408 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.088649 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:41.088729 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:41.089260 (+ 531us) server_negotiation.cc:299] Negotiation successful
0927 23:02:41.089455 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260927 23:02:41.089748 23715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.078635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:38227 (local address 127.4.74.130:47097)
0927 23:02:41.080317 (+ 1682us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:41.080343 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:41.080948 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:41.081524 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:41.081528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:41.081551 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:41.082038 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:41.082058 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.083677 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.083684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:41.086396 (+ 2712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:41.086434 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.088098 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.088107 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:41.088351 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:41.088396 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 23:02:41.088569 (+ 173us) client_negotiation.cc:240] Negotiation successful
0927 23:02:41.088780 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1409,"mutex_wait_us":37,"thread_start_us":694,"threads_started":1}
I20260927 23:02:41.089787 23716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.078890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44623 (local address 127.4.74.130:45993)
0927 23:02:41.080545 (+ 1655us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:41.080566 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:41.080745 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:41.081258 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:41.081263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:41.081299 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:41.081850 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:41.081863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.083352 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.083358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:41.086521 (+ 3163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:41.086545 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.088734 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.088742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:41.088980 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:41.089022 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 23:02:41.089164 (+ 142us) client_negotiation.cc:240] Negotiation successful
0927 23:02:41.089341 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1381,"mutex_wait_us":73,"thread_start_us":1095,"threads_started":1}
I20260927 23:02:41.091674 23643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129126bac13946f1810454f153dac621" candidate_uuid: "961991ac868847399fb630a0d497b934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" is_pre_election: true
I20260927 23:02:41.091614 23381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129126bac13946f1810454f153dac621" candidate_uuid: "961991ac868847399fb630a0d497b934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9528fc3c3e9e4c95bd406813fd325685" is_pre_election: true
I20260927 23:02:41.092371 23643 raft_consensus.cc:2454] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961991ac868847399fb630a0d497b934 in term 0.
I20260927 23:02:41.092552 23381 raft_consensus.cc:2454] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 961991ac868847399fb630a0d497b934 in term 0.
I20260927 23:02:41.093505 23448 leader_election.cc:302] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7669f278389b434ebb1fdfdd1cbe69e2, 961991ac868847399fb630a0d497b934; no voters:
I20260927 23:02:41.094053 23712 raft_consensus.cc:2781] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:02:41.094244 23712 raft_consensus.cc:482] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:02:41.094370 23712 raft_consensus.cc:3037] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:41.096928 23712 raft_consensus.cc:504] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961991ac868847399fb630a0d497b934" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43425 } } peers { permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 } } peers { permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 } }
I20260927 23:02:41.097646 23712 leader_election.cc:288] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [CANDIDATE]: Term 1 election: Requested vote from peers 7669f278389b434ebb1fdfdd1cbe69e2 (127.4.74.131:38227), 9528fc3c3e9e4c95bd406813fd325685 (127.4.74.129:44623)
I20260927 23:02:41.098532 23643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129126bac13946f1810454f153dac621" candidate_uuid: "961991ac868847399fb630a0d497b934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7669f278389b434ebb1fdfdd1cbe69e2"
I20260927 23:02:41.098827 23643 raft_consensus.cc:3037] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:41.098842 23381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "129126bac13946f1810454f153dac621" candidate_uuid: "961991ac868847399fb630a0d497b934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9528fc3c3e9e4c95bd406813fd325685"
I20260927 23:02:41.099229 23381 raft_consensus.cc:3037] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:41.102183 23643 raft_consensus.cc:2454] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961991ac868847399fb630a0d497b934 in term 1.
I20260927 23:02:41.102905 23448 leader_election.cc:302] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7669f278389b434ebb1fdfdd1cbe69e2, 961991ac868847399fb630a0d497b934; no voters:
I20260927 23:02:41.103237 23381 raft_consensus.cc:2454] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 961991ac868847399fb630a0d497b934 in term 1.
I20260927 23:02:41.103454 23712 raft_consensus.cc:2781] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:41.104223 23712 raft_consensus.cc:686] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 1 LEADER]: Becoming Leader. State: Replica: 961991ac868847399fb630a0d497b934, State: Running, Role: LEADER
I20260927 23:02:41.105068 23712 consensus_queue.cc:237] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961991ac868847399fb630a0d497b934" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43425 } } peers { permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 } } peers { permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 } }
I20260927 23:02:41.112285 23238 catalog_manager.cc:5495] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 reported cstate change: term changed from 0 to 1, leader changed from <none> to 961991ac868847399fb630a0d497b934 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "961991ac868847399fb630a0d497b934" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "961991ac868847399fb630a0d497b934" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 } health_report { overall_health: UNKNOWN } } }
I20260927 23:02:41.157476 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.150574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.4.74.130:43425)
0927 23:02:41.150862 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:41.150867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:41.150887 (+ 20us) server_negotiation.cc:407] Connection header received
0927 23:02:41.151196 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:41.151203 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:41.151383 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:41.151534 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:02:41.152344 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.153186 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.154829 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.155273 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.155430 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:41.156189 (+ 759us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:02:41.156776 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:02:41.156978 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:41.157157 (+ 179us) server_negotiation.cc:299] Negotiation successful
0927 23:02:41.157203 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260927 23:02:41.169349 23643 raft_consensus.cc:1261] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2 [term 1 FOLLOWER]: Refusing update from remote peer 961991ac868847399fb630a0d497b934: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:41.169655 23381 raft_consensus.cc:1261] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685 [term 1 FOLLOWER]: Refusing update from remote peer 961991ac868847399fb630a0d497b934: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:41.170771 23717 consensus_queue.cc:1035] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7669f278389b434ebb1fdfdd1cbe69e2" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 38227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:41.171206 23712 consensus_queue.cc:1035] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9528fc3c3e9e4c95bd406813fd325685" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 44623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:41.212697 23239 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:41.213197 23239 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:41.219762 23239 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8bc8c7e625014f43b02e765d8f6bce6d: Sending DeleteTablet for 3 replicas of tablet 129126bac13946f1810454f153dac621
I20260927 23:02:41.221537 23623 tablet_service.cc:1514] Processing DeleteTablet for tablet 129126bac13946f1810454f153dac621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:41 UTC) from {username='slave'} at 127.0.0.1:36396
I20260927 23:02:41.221544 23492 tablet_service.cc:1514] Processing DeleteTablet for tablet 129126bac13946f1810454f153dac621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:41 UTC) from {username='slave'} at 127.0.0.1:55310
I20260927 23:02:41.223238 23361 tablet_service.cc:1514] Processing DeleteTablet for tablet 129126bac13946f1810454f153dac621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:41 UTC) from {username='slave'} at 127.0.0.1:41568
I20260927 23:02:41.224913 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 23299
I20260927 23:02:41.233799 23731 tablet_replica.cc:330] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: stopping tablet replica
I20260927 23:02:41.234483 23731 raft_consensus.cc:2229] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:02:41.238306 23731 raft_consensus.cc:2258] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:41.239261 23732 tablet_replica.cc:330] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685: stopping tablet replica
I20260927 23:02:41.239991 23732 raft_consensus.cc:2229] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:41.240619 23732 raft_consensus.cc:2258] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:41.242866 23730 tablet_replica.cc:330] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: stopping tablet replica
I20260927 23:02:41.243424 23730 raft_consensus.cc:2229] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:41.243470 23732 ts_tablet_manager.cc:1905] T 129126bac13946f1810454f153dac621 P 9528fc3c3e9e4c95bd406813fd325685: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:41.244037 23730 raft_consensus.cc:2258] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:41.245158 23731 ts_tablet_manager.cc:1905] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:41.248287 23730 ts_tablet_manager.cc:1905] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:41.253475 23731 ts_tablet_manager.cc:1918] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:41.253803 23731 log.cc:1196] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/129126bac13946f1810454f153dac621
I20260927 23:02:41.254470 23703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.226993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41574 (local address 127.4.74.129:44623)
0927 23:02:41.227268 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:41.227279 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:41.239459 (+ 12180us) server_negotiation.cc:407] Connection header received
0927 23:02:41.240819 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:41.240826 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:41.240980 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:41.241071 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:41.241886 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.243143 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.245299 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.246082 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.246212 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:41.251969 (+ 5757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:41.251985 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:41.252010 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:41.253116 (+ 1106us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:41.253208 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:41.253247 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:41.253385 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:41.253836 (+ 451us) server_negotiation.cc:299] Negotiation successful
0927 23:02:41.254012 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260927 23:02:41.254829 23731 ts_tablet_manager.cc:1939] T 129126bac13946f1810454f153dac621 P 961991ac868847399fb630a0d497b934: Deleting consensus metadata
I20260927 23:02:41.256147 23421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:41.257203 23226 catalog_manager.cc:4841] TS 961991ac868847399fb630a0d497b934 (127.4.74.130:43425): tablet 129126bac13946f1810454f153dac621 (table test-table [id=9101e929198b419ab1801f6293491373]) successfully deleted
I20260927 23:02:41.257274 23730 ts_tablet_manager.cc:1918] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:41.257656 23730 log.cc:1196] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/129126bac13946f1810454f153dac621
I20260927 23:02:41.258315 23730 ts_tablet_manager.cc:1939] T 129126bac13946f1810454f153dac621 P 7669f278389b434ebb1fdfdd1cbe69e2: Deleting consensus metadata
I20260927 23:02:41.260691 23227 catalog_manager.cc:4841] TS 7669f278389b434ebb1fdfdd1cbe69e2 (127.4.74.131:38227): tablet 129126bac13946f1810454f153dac621 (table test-table [id=9101e929198b419ab1801f6293491373]) successfully deleted
I20260927 23:02:41.355325 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 23299
W20260927 23:02:41.374571 23225 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.74.129:44623 (error 108)
I20260927 23:02:41.374967 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 23430
I20260927 23:02:41.377316 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.376788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44623 (local address 127.0.0.1:41586)
0927 23:02:41.376969 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:41.377106 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260927 23:02:41.377626 23225 catalog_manager.cc:4584] TS 9528fc3c3e9e4c95bd406813fd325685 (127.4.74.129:44623): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 129126bac13946f1810454f153dac621: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44623: connect: Connection refused (error 111)
I20260927 23:02:41.385530 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.376925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55332 (local address 127.4.74.130:43425)
0927 23:02:41.377126 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:41.377133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:41.377158 (+ 25us) server_negotiation.cc:407] Connection header received
0927 23:02:41.377223 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:41.377229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:41.377375 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:41.377497 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:41.378397 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.379317 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.381445 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.381984 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.382115 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:41.383757 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:41.383766 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:41.383782 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:41.384612 (+ 830us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:41.384678 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:41.384703 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:41.384834 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:41.385027 (+ 193us) server_negotiation.cc:299] Negotiation successful
0927 23:02:41.385132 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 23:02:41.386551 23552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:41.431986 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.431161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44623 (local address 127.0.0.1:41590)
0927 23:02:41.431527 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:41.431755 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260927 23:02:41.433377 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.432953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44623 (local address 127.0.0.1:41600)
0927 23:02:41.433153 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:41.433235 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 23:02:41.494263 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.493623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44623 (local address 127.0.0.1:41604)
0927 23:02:41.493913 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:41.494050 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260927 23:02:41.495605 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.495127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44623 (local address 127.0.0.1:41618)
0927 23:02:41.495296 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:41.495381 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260927 23:02:41.507788 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 23430
I20260927 23:02:41.530642 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 23561
I20260927 23:02:41.538916 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.531812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36398 (local address 127.4.74.131:38227)
0927 23:02:41.532048 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:41.532057 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:41.532084 (+ 27us) server_negotiation.cc:407] Connection header received
0927 23:02:41.532369 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:41.532374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:41.532479 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:41.532615 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:41.533074 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.533857 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.535371 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:41.535926 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:41.536086 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:41.537366 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:41.537375 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:41.537391 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:41.538031 (+ 640us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:41.538082 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:41.538108 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:41.538259 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:41.538420 (+ 161us) server_negotiation.cc:299] Negotiation successful
0927 23:02:41.538542 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260927 23:02:41.539954 23683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:41.577205 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.576317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44623 (local address 127.0.0.1:41622)
0927 23:02:41.576713 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:41.576940 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260927 23:02:41.578860 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.578242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44623 (local address 127.0.0.1:41626)
0927 23:02:41.578539 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:41.578667 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260927 23:02:41.645721 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 23561
I20260927 23:02:41.665777 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 23209
I20260927 23:02:41.667225 23270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:41.789834 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:41.788864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:44623 (local address 127.0.0.1:41634)
0927 23:02:41.789296 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:41.789480 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260927 23:02:41.790704 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 23209
2026-09-27T23:02:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3221 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:02:41 dist-test-slave-5pd3 krb5kdc[23741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:02:41 dist-test-slave-5pd3 krb5kdc[23741](info): set up 2 sockets
Sep 27 23:02:41 dist-test-slave-5pd3 krb5kdc[23741](info): commencing operation
krb5kdc: starting...
W20260927 23:02:42.960127 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.007s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:02:43 dist-test-slave-5pd3 krb5kdc[23741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:02:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:02:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:02:43.268682 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:46395
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:46395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:43.711071 23754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:43.711505 23754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:43.711723 23754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:43.720709 23754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:43.720885 23754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:43.721011 23754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:43.721138 23754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:43.721210 23754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:43.733250 23754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:46395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:46395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:43.735504 23754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:43.737962 23754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:43.753242 23759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:43.753172 23762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:43.753193 23760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:43.754357 23754 server_base.cc:1034] running on GCE node
I20260927 23:02:43.755420 23754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:43.757526 23754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:43.758816 23754 hybrid_clock.cc:648] HybridClock initialized: now 1790550163758747 us; error 66 us; skew 500 ppm
I20260927 23:02:43.759351 23754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:43 dist-test-slave-5pd3 krb5kdc[23741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550163, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:43.771414 23754 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:02:43.773693 23754 webserver.cc:460] Webserver started at http://127.4.74.190:38167/ using document root <none> and password file <none>
I20260927 23:02:43.774710 23754 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:43.774849 23754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:43.775260 23754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:43.779317 23754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "0522dfc58603445abfddbe41ccdcda74"
format_stamp: "Formatted at 2026-09-27 23:02:43 on dist-test-slave-5pd3"
server_key: "ed9d6de618c8fa63b98a1e2cffe9aee2"
server_key_iv: "12512fc5a51d848ed6fbc8c3a56241ad"
server_key_version: "encryptionkey@0"
I20260927 23:02:43.780799 23754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "0522dfc58603445abfddbe41ccdcda74"
format_stamp: "Formatted at 2026-09-27 23:02:43 on dist-test-slave-5pd3"
server_key: "ed9d6de618c8fa63b98a1e2cffe9aee2"
server_key_iv: "12512fc5a51d848ed6fbc8c3a56241ad"
server_key_version: "encryptionkey@0"
I20260927 23:02:43.788407 23754 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260927 23:02:43.793387 23769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:43.795454 23754 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260927 23:02:43.795820 23754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "0522dfc58603445abfddbe41ccdcda74"
format_stamp: "Formatted at 2026-09-27 23:02:43 on dist-test-slave-5pd3"
server_key: "ed9d6de618c8fa63b98a1e2cffe9aee2"
server_key_iv: "12512fc5a51d848ed6fbc8c3a56241ad"
server_key_version: "encryptionkey@0"
I20260927 23:02:43.796110 23754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:43.835935 23754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:43.842250 23754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:43.842684 23754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:43.867785 23754 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:46395
I20260927 23:02:43.867894 23820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:46395 every 8 connection(s)
I20260927 23:02:43.869421 23754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:43.873649 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 23754
I20260927 23:02:43.873898 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:02:43.874396 4394 external_mini_cluster.cc:1428] Setting key c7b747cc32e2d04993a03406d5c384c8
I20260927 23:02:43.875123 23821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 23:02:43 dist-test-slave-5pd3 krb5kdc[23741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:43.894979 23821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74: Bootstrap starting.
I20260927 23:02:43.901337 23821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:43.902956 23821 log.cc:824] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:43.906644 23824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:43.876680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55856 (local address 127.4.74.190:46395)
0927 23:02:43.877629 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:43.877643 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:43.877693 (+ 50us) server_negotiation.cc:407] Connection header received
0927 23:02:43.879522 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:43.879609 (+ 87us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:43.880123 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:43.880554 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:43.883016 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:43.884790 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:43.886909 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:43.887909 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:43.888098 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:43.894803 (+ 6705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:43.894840 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:43.894869 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:43.894960 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:43.900450 (+ 5490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:43.901701 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:43.901711 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:43.901719 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:43.902108 (+ 389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:43.902555 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:43.902560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:43.902564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:43.903393 (+ 829us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:43.903782 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:43.904310 (+ 528us) server_negotiation.cc:299] Negotiation successful
0927 23:02:43.904894 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":425,"threads_started":1}
I20260927 23:02:43.907627 23821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74: No bootstrap required, opened a new log
I20260927 23:02:43.917404 23821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522dfc58603445abfddbe41ccdcda74" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46395 } }
I20260927 23:02:43.917872 23821 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:43.917944 23821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0522dfc58603445abfddbe41ccdcda74, State: Initialized, Role: FOLLOWER
I20260927 23:02:43.918569 23821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522dfc58603445abfddbe41ccdcda74" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46395 } }
I20260927 23:02:43.918792 23821 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:43.918900 23821 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:43.919039 23821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:43.921482 23821 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522dfc58603445abfddbe41ccdcda74" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46395 } }
I20260927 23:02:43.922165 23821 leader_election.cc:302] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0522dfc58603445abfddbe41ccdcda74; no voters:
I20260927 23:02:43.922995 23821 leader_election.cc:288] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:02:43.923254 23826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:43.924217 23826 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [term 1 LEADER]: Becoming Leader. State: Replica: 0522dfc58603445abfddbe41ccdcda74, State: Running, Role: LEADER
I20260927 23:02:43.925042 23826 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522dfc58603445abfddbe41ccdcda74" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46395 } }
I20260927 23:02:43.925589 23821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:43.929009 23827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0522dfc58603445abfddbe41ccdcda74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522dfc58603445abfddbe41ccdcda74" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46395 } } }
I20260927 23:02:43.929772 23827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:43.929020 23828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0522dfc58603445abfddbe41ccdcda74. Latest consensus state: current_term: 1 leader_uuid: "0522dfc58603445abfddbe41ccdcda74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522dfc58603445abfddbe41ccdcda74" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 46395 } } }
I20260927 23:02:43.930202 23828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:43.935818 23835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:43.941701 23835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:43.952804 23835 catalog_manager.cc:1347] Generated new cluster ID: 16966e0eb06d4b78b6c9d1dd0482987a
I20260927 23:02:43.952943 23835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:43.968883 23835 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:02:43.970621 23835 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:43.988008 23835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74: Generated new TSK 0
I20260927 23:02:43.989090 23835 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
Entry for principal kudu/127.4.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab.
I20260927 23:02:44.060920 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46395
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:44.453388 23847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:44.453716 23847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:44.453824 23847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:44.461365 23847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:44.461501 23847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:44.461622 23847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:02:44.472024 23847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.129.keytab
--principal=kudu/127.4.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:44.473547 23847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:44.474972 23847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:44.489604 23853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:44.489941 23855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:44.490242 23852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:44.491990 23847 server_base.cc:1034] running on GCE node
I20260927 23:02:44.493080 23847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:44.494601 23847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:44.495919 23847 hybrid_clock.cc:648] HybridClock initialized: now 1790550164495820 us; error 80 us; skew 500 ppm
I20260927 23:02:44.496450 23847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:44 dist-test-slave-5pd3 krb5kdc[23741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550164, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:44.501374 23847 init.cc:377] Logged in from keytab as kudu/127.4.74.129@KRBTEST.COM (short username kudu)
I20260927 23:02:44.503803 23847 webserver.cc:460] Webserver started at http://127.4.74.129:35677/ using document root <none> and password file <none>
I20260927 23:02:44.504786 23847 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:44.504942 23847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:44.505350 23847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:44.509423 23847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "715eb97b8a85428d81aa51b9a72d63e6"
format_stamp: "Formatted at 2026-09-27 23:02:44 on dist-test-slave-5pd3"
server_key: "72ade5255cb20582b8491fdf566faf25"
server_key_iv: "ab1a35d9088320c24055b861cec5503e"
server_key_version: "encryptionkey@0"
I20260927 23:02:44.510526 23847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "715eb97b8a85428d81aa51b9a72d63e6"
format_stamp: "Formatted at 2026-09-27 23:02:44 on dist-test-slave-5pd3"
server_key: "72ade5255cb20582b8491fdf566faf25"
server_key_iv: "ab1a35d9088320c24055b861cec5503e"
server_key_version: "encryptionkey@0"
I20260927 23:02:44.517827 23847 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260927 23:02:44.523674 23862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:44.526060 23847 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260927 23:02:44.526360 23847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "715eb97b8a85428d81aa51b9a72d63e6"
format_stamp: "Formatted at 2026-09-27 23:02:44 on dist-test-slave-5pd3"
server_key: "72ade5255cb20582b8491fdf566faf25"
server_key_iv: "ab1a35d9088320c24055b861cec5503e"
server_key_version: "encryptionkey@0"
I20260927 23:02:44.526703 23847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:44.552001 23847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:44.557572 23847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:44.557952 23847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:44.559301 23847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:44.561092 23847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:44.561218 23847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:44.561393 23847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:44.561470 23847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:44.604912 23847 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:36923
I20260927 23:02:44.604964 23974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:36923 every 8 connection(s)
I20260927 23:02:44.606822 23847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:02:44.610641 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 23847
I20260927 23:02:44.610884 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 23:02:44.611670 4394 external_mini_cluster.cc:1428] Setting key 5887cf0f76982fa8926335f57c45850f
Sep 27 23:02:44 dist-test-slave-5pd3 krb5kdc[23741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550164, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.129@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:44.635964 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:44.610380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:35157 (local address 127.4.74.190:46395)
0927 23:02:44.611054 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:44.611063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:44.612152 (+ 1089us) server_negotiation.cc:407] Connection header received
0927 23:02:44.614915 (+ 2763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:44.614922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:44.615039 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:44.615216 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:44.617161 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:44.618532 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:44.620675 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:44.621489 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:44.621739 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:44.626603 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:44.626613 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:44.626621 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:44.626706 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:44.630245 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:44.631784 (+ 1539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:44.631790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:44.631814 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:44.632039 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:44.632739 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:44.632744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:44.632746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:44.633200 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:44.633352 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:44.634984 (+ 1632us) server_negotiation.cc:299] Negotiation successful
0927 23:02:44.635588 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":330,"threads_started":1}
I20260927 23:02:44.637226 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:44.610966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46395 (local address 127.4.74.129:35157)
0927 23:02:44.611929 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:44.611979 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:44.614431 (+ 2452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:44.615485 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:44.615494 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:44.616207 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:44.616864 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:44.616883 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:44.618803 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:44.618808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:44.620424 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:44.620441 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:44.621912 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:44.621919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:44.622198 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:44.623285 (+ 1087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:02:44.623318 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:44.626148 (+ 2830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:44.630536 (+ 4388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:44.630553 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:44.630579 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:44.631489 (+ 910us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:44.632224 (+ 735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:44.632232 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:44.632239 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:44.632553 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:44.633381 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:44.633392 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:44.633936 (+ 544us) client_negotiation.cc:769] Sending connection context
0927 23:02:44.634601 (+ 665us) client_negotiation.cc:240] Negotiation successful
0927 23:02:44.635403 (+ 802us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":395,"threads_started":1}
I20260927 23:02:44.640600 23975 heartbeater.cc:347] Connected to a master server at 127.4.74.190:46395
I20260927 23:02:44.641194 23975 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:44.642320 23975 heartbeater.cc:511] Master 127.4.74.190:46395 requested a full tablet report, sending...
I20260927 23:02:44.645574 23786 ts_manager.cc:194] Registered new tserver with Master: 715eb97b8a85428d81aa51b9a72d63e6 (127.4.74.129:36923)
WARNING: no policy specified for kudu/127.4.74.130@KRBTEST.COM; defaulting to no policy
I20260927 23:02:44.647663 23786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.129@KRBTEST.COM'} at 127.4.74.129:35157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
Entry for principal kudu/127.4.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab.
I20260927 23:02:44.684762 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46395
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:45.042207 23981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:45.042532 23981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:45.042630 23981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:45.050217 23981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:45.050371 23981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:45.050513 23981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:02:45.060689 23981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.130.keytab
--principal=kudu/127.4.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:45.061996 23981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:45.063868 23981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:45.079293 23986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:45.079535 23989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:45.080428 23987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:45.081452 23981 server_base.cc:1034] running on GCE node
I20260927 23:02:45.082355 23981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:45.083513 23981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:45.084949 23981 hybrid_clock.cc:648] HybridClock initialized: now 1790550165084863 us; error 77 us; skew 500 ppm
I20260927 23:02:45.085467 23981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:45 dist-test-slave-5pd3 krb5kdc[23741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550165, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:45.089444 23981 init.cc:377] Logged in from keytab as kudu/127.4.74.130@KRBTEST.COM (short username kudu)
I20260927 23:02:45.091280 23981 webserver.cc:460] Webserver started at http://127.4.74.130:46083/ using document root <none> and password file <none>
I20260927 23:02:45.092137 23981 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:45.092295 23981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:45.092695 23981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:45.096199 23981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "e1c2c85134554605b9628bcc6f341daa"
format_stamp: "Formatted at 2026-09-27 23:02:45 on dist-test-slave-5pd3"
server_key: "542d852979ffc1d2aeec2814af5c1a3f"
server_key_iv: "2c73bcf6777dfb0b8af9b983a0465ea3"
server_key_version: "encryptionkey@0"
I20260927 23:02:45.097100 23981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "e1c2c85134554605b9628bcc6f341daa"
format_stamp: "Formatted at 2026-09-27 23:02:45 on dist-test-slave-5pd3"
server_key: "542d852979ffc1d2aeec2814af5c1a3f"
server_key_iv: "2c73bcf6777dfb0b8af9b983a0465ea3"
server_key_version: "encryptionkey@0"
I20260927 23:02:45.103930 23981 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260927 23:02:45.108423 23996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:45.110347 23981 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:02:45.110602 23981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "e1c2c85134554605b9628bcc6f341daa"
format_stamp: "Formatted at 2026-09-27 23:02:45 on dist-test-slave-5pd3"
server_key: "542d852979ffc1d2aeec2814af5c1a3f"
server_key_iv: "2c73bcf6777dfb0b8af9b983a0465ea3"
server_key_version: "encryptionkey@0"
I20260927 23:02:45.110859 23981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:45.132498 23981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:45.137771 23981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:45.138049 23981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:45.139252 23981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:45.140959 23981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:45.141044 23981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:45.141141 23981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:45.141193 23981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:45.181398 23981 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:38745
I20260927 23:02:45.181540 24108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:38745 every 8 connection(s)
I20260927 23:02:45.182837 23981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:02:45.185302 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 23981
I20260927 23:02:45.185495 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 23:02:45.186033 4394 external_mini_cluster.cc:1428] Setting key 7e07af0353d5ebf884c6023e85763015
Sep 27 23:02:45 dist-test-slave-5pd3 krb5kdc[23741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550165, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.130@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:45.211265 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.186612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:38757 (local address 127.4.74.190:46395)
0927 23:02:45.187601 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:45.187608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:45.188951 (+ 1343us) server_negotiation.cc:407] Connection header received
0927 23:02:45.191383 (+ 2432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:45.191391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:45.191506 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:45.191704 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:45.193879 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.195149 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.198147 (+ 2998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.198675 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.198815 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.204690 (+ 5875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:45.204696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:45.204700 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:45.204745 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:45.207773 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:45.208695 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:45.208703 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:45.208706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:45.208848 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:45.209327 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:45.209337 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:45.209339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:45.209652 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:45.209827 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:45.210601 (+ 774us) server_negotiation.cc:299] Negotiation successful
0927 23:02:45.210851 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":672,"threads_started":1}
I20260927 23:02:45.212369 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.187536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46395 (local address 127.4.74.130:38757)
0927 23:02:45.188711 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:45.188769 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:45.191006 (+ 2237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:45.191997 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:45.192012 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:45.192765 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:45.193612 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:45.193638 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.195442 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.195453 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:45.197869 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:45.197902 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.199024 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.199035 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:45.199446 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.200543 (+ 1097us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:02:45.200594 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:45.204352 (+ 3758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:45.207987 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:45.208000 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:45.208028 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:45.208541 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:45.208973 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:45.208978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:45.208982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:45.209195 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:45.209804 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:45.209812 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:45.210218 (+ 406us) client_negotiation.cc:769] Sending connection context
0927 23:02:45.210507 (+ 289us) client_negotiation.cc:240] Negotiation successful
0927 23:02:45.211137 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":468,"threads_started":1}
I20260927 23:02:45.215876 24109 heartbeater.cc:347] Connected to a master server at 127.4.74.190:46395
I20260927 23:02:45.216426 24109 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.74.131@KRBTEST.COM; defaulting to no policy
I20260927 23:02:45.217815 24109 heartbeater.cc:511] Master 127.4.74.190:46395 requested a full tablet report, sending...
I20260927 23:02:45.220191 23786 ts_manager.cc:194] Registered new tserver with Master: e1c2c85134554605b9628bcc6f341daa (127.4.74.130:38745)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.131@KRBTEST.COM" created.
I20260927 23:02:45.221256 23786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.130@KRBTEST.COM'} at 127.4.74.130:38757
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
Entry for principal kudu/127.4.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab.
I20260927 23:02:45.260931 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46395
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:36407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:45.606566 24115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:45.606950 24115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:45.607115 24115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:45.615085 24115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:45.615286 24115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:45.615530 24115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:02:45.626351 24115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:36407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.131.keytab
--principal=kudu/127.4.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:46395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:45.627925 24115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:45.629303 24115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:45.645750 24120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:45.645949 24123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:45.645953 24121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:45.646831 24115 server_base.cc:1034] running on GCE node
I20260927 23:02:45.647789 24115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:45.649067 24115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:45.650326 24115 hybrid_clock.cc:648] HybridClock initialized: now 1790550165650259 us; error 51 us; skew 500 ppm
I20260927 23:02:45.650835 24115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:45.652457 23975 heartbeater.cc:503] Master 127.4.74.190:46395 was elected leader, sending a full tablet report...
Sep 27 23:02:45 dist-test-slave-5pd3 krb5kdc[23741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550165, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:45.655861 24115 init.cc:377] Logged in from keytab as kudu/127.4.74.131@KRBTEST.COM (short username kudu)
I20260927 23:02:45.657943 24115 webserver.cc:460] Webserver started at http://127.4.74.131:45743/ using document root <none> and password file <none>
I20260927 23:02:45.658723 24115 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:45.658843 24115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:45.659140 24115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:45.662410 24115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "4eea257b828840cbba639591a4134b45"
format_stamp: "Formatted at 2026-09-27 23:02:45 on dist-test-slave-5pd3"
server_key: "9d4cbbf9101b199124bdd8ba3345a7e4"
server_key_iv: "ff71c315f7bada6df33a6c2b829894d1"
server_key_version: "encryptionkey@0"
I20260927 23:02:45.663194 24115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "4eea257b828840cbba639591a4134b45"
format_stamp: "Formatted at 2026-09-27 23:02:45 on dist-test-slave-5pd3"
server_key: "9d4cbbf9101b199124bdd8ba3345a7e4"
server_key_iv: "ff71c315f7bada6df33a6c2b829894d1"
server_key_version: "encryptionkey@0"
I20260927 23:02:45.669198 24115 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260927 23:02:45.673734 24130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:45.675858 24115 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 23:02:45.676129 24115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "4eea257b828840cbba639591a4134b45"
format_stamp: "Formatted at 2026-09-27 23:02:45 on dist-test-slave-5pd3"
server_key: "9d4cbbf9101b199124bdd8ba3345a7e4"
server_key_iv: "ff71c315f7bada6df33a6c2b829894d1"
server_key_version: "encryptionkey@0"
I20260927 23:02:45.676400 24115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:45.706161 24115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:45.711485 24115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:45.711915 24115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:45.713263 24115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:45.715082 24115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:45.715175 24115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:45.715279 24115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:45.715335 24115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:45.758436 24115 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:37805
I20260927 23:02:45.758569 24242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:37805 every 8 connection(s)
I20260927 23:02:45.759922 24115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:02:45.763725 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 24115
I20260927 23:02:45.763983 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 23:02:45.764642 4394 external_mini_cluster.cc:1428] Setting key b76691d33a3133bb0e97f290196f8dce
Sep 27 23:02:45 dist-test-slave-5pd3 krb5kdc[23741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550165, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.131@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:02:45.786691 24245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.763999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:37965 (local address 127.4.74.190:46395)
0927 23:02:45.764624 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:45.764629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:45.765821 (+ 1192us) server_negotiation.cc:407] Connection header received
0927 23:02:45.768680 (+ 2859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:45.768686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:45.768824 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:45.768975 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:45.770666 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.771760 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.773523 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.774085 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.774232 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.778704 (+ 4472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:45.778712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:45.778716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:45.778798 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:45.783035 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:45.784090 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:45.784095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:45.784097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:45.784256 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:45.784720 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:45.784724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:45.784727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:45.785112 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:45.785268 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:45.786078 (+ 810us) server_negotiation.cc:299] Negotiation successful
0927 23:02:45.786342 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":355,"threads_started":1}
I20260927 23:02:45.787664 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.764404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:46395 (local address 127.4.74.131:37965)
0927 23:02:45.765594 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:45.765672 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:45.768278 (+ 2606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:45.769207 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:45.769215 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:45.769850 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:45.770462 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:45.770480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.771939 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.771943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:45.773328 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:45.773342 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.774344 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.774349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:45.774560 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.775392 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:02:45.775417 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:45.778353 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:45.783265 (+ 4912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:45.783274 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:45.783287 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:45.783940 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:45.784368 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:02:45.784371 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:02:45.784374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:02:45.784562 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:02:45.785254 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:45.785260 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:45.785627 (+ 367us) client_negotiation.cc:769] Sending connection context
0927 23:02:45.785891 (+ 264us) client_negotiation.cc:240] Negotiation successful
0927 23:02:45.786350 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":507,"threads_started":1}
I20260927 23:02:45.789813 24243 heartbeater.cc:347] Connected to a master server at 127.4.74.190:46395
I20260927 23:02:45.790226 24243 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:45.791049 24243 heartbeater.cc:511] Master 127.4.74.190:46395 requested a full tablet report, sending...
I20260927 23:02:45.793133 23786 ts_manager.cc:194] Registered new tserver with Master: 4eea257b828840cbba639591a4134b45 (127.4.74.131:37805)
I20260927 23:02:45.794464 23786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.74.131@KRBTEST.COM'} at 127.4.74.131:37965
I20260927 23:02:45.795353 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:02:45.815491 24245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.800722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45956 (local address 127.4.74.190:46395)
0927 23:02:45.801108 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:45.801118 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:45.801400 (+ 282us) server_negotiation.cc:407] Connection header received
0927 23:02:45.801579 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:45.801584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:45.801669 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:45.801776 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:45.803437 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.804329 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.805705 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.806202 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.806318 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.808996 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:45.809001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:45.809005 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:45.809059 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:45.812147 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:45.813024 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:45.813049 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:45.813052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:45.813200 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:45.813648 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:45.813651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:45.813653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:45.814094 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:45.814323 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:45.814710 (+ 387us) server_negotiation.cc:299] Negotiation successful
0927 23:02:45.815085 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260927 23:02:45.821408 23786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:02:45.829336 23786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:02:45.875591 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.846204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:37805 (local address 127.0.0.1:41092)
0927 23:02:45.860644 (+ 14440us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:45.860734 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:45.861105 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:45.865971 (+ 4866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:45.865980 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:45.866029 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:45.866848 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:45.866871 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.869280 (+ 2409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.869286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:45.871754 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:45.871774 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.874326 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.874332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:45.874554 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.874598 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 23:02:45.874724 (+ 126us) client_negotiation.cc:240] Negotiation successful
0927 23:02:45.875039 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14028,"mutex_wait_us":1,"thread_start_us":14691,"threads_started":1}
I20260927 23:02:45.875886 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.846424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48646 (local address 127.4.74.129:36923)
0927 23:02:45.860859 (+ 14435us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:45.860868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:45.861586 (+ 718us) server_negotiation.cc:407] Connection header received
0927 23:02:45.861719 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:45.861732 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:45.862027 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:45.862257 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:45.863532 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.866097 (+ 2565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.870877 (+ 4780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.873331 (+ 2454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.873535 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.873848 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:45.875003 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0927 23:02:45.875293 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14235,"thread_start_us":14035,"threads_started":1}
I20260927 23:02:45.877024 24257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.846242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:36923 (local address 127.0.0.1:48646)
0927 23:02:45.861195 (+ 14953us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:45.861218 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:45.861424 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:45.862663 (+ 1239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:45.862676 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:45.862748 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:45.863263 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:45.863282 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.866345 (+ 3063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.866356 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:45.870522 (+ 4166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:45.870556 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.874069 (+ 3513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.874081 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:45.874436 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.874532 (+ 96us) client_negotiation.cc:769] Sending connection context
0927 23:02:45.874764 (+ 232us) client_negotiation.cc:240] Negotiation successful
0927 23:02:45.876382 (+ 1618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14780,"mutex_wait_us":1291,"thread_start_us":18377,"threads_started":1}
I20260927 23:02:45.877130 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.846984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.4.74.131:37805)
0927 23:02:45.859905 (+ 12921us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:45.859921 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:45.861474 (+ 1553us) server_negotiation.cc:407] Connection header received
0927 23:02:45.861623 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:45.861635 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:45.865114 (+ 3479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:45.865449 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:45.867182 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.869084 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.871984 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.873722 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.873847 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.874108 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:45.876352 (+ 2244us) server_negotiation.cc:299] Negotiation successful
0927 23:02:45.876708 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12585,"thread_start_us":12952,"threads_started":1}
I20260927 23:02:45.883147 23910 tablet_service.cc:1467] Processing CreateTablet for tablet 4047f4f5a01d43219b89e34ac19a1756 (DEFAULT_TABLE table=test-table [id=3704406ff9f941e5bd0aa3f8f77af402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:45.885876 23910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4047f4f5a01d43219b89e34ac19a1756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:45.888175 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.862204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:38745 (local address 127.0.0.1:42420)
0927 23:02:45.875934 (+ 13730us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:45.875958 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:45.876123 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:45.878970 (+ 2847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:45.878976 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:45.879009 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:45.879232 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:45.879244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.881564 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.881569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:45.883979 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:45.883999 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.886655 (+ 2656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.886662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:45.887439 (+ 777us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.887495 (+ 56us) client_negotiation.cc:769] Sending connection context
0927 23:02:45.887643 (+ 148us) client_negotiation.cc:240] Negotiation successful
0927 23:02:45.887833 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13596,"threads_started":1}
I20260927 23:02:45.889104 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:45.862279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42420 (local address 127.4.74.130:38745)
0927 23:02:45.877905 (+ 15626us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:45.877915 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:45.877959 (+ 44us) server_negotiation.cc:407] Connection header received
0927 23:02:45.878098 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:45.878109 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:45.878445 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:45.878707 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:45.879403 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.881385 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.884233 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:45.886015 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:45.886221 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:45.886492 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:45.888506 (+ 2014us) server_negotiation.cc:299] Negotiation successful
0927 23:02:45.888755 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15448,"thread_start_us":15257,"threads_started":1}
I20260927 23:02:45.895879 24044 tablet_service.cc:1467] Processing CreateTablet for tablet 4047f4f5a01d43219b89e34ac19a1756 (DEFAULT_TABLE table=test-table [id=3704406ff9f941e5bd0aa3f8f77af402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:45.895879 24178 tablet_service.cc:1467] Processing CreateTablet for tablet 4047f4f5a01d43219b89e34ac19a1756 (DEFAULT_TABLE table=test-table [id=3704406ff9f941e5bd0aa3f8f77af402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:45.897861 24178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4047f4f5a01d43219b89e34ac19a1756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:45.897861 24044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4047f4f5a01d43219b89e34ac19a1756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:45.908002 24262 tablet_bootstrap.cc:492] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: Bootstrap starting.
I20260927 23:02:45.913007 24262 tablet_bootstrap.cc:654] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:45.914254 24262 log.cc:824] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:45.919384 24264 tablet_bootstrap.cc:492] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: Bootstrap starting.
I20260927 23:02:45.922780 24265 tablet_bootstrap.cc:492] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: Bootstrap starting.
I20260927 23:02:45.925295 24264 tablet_bootstrap.cc:654] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:45.925375 24262 tablet_bootstrap.cc:492] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: No bootstrap required, opened a new log
I20260927 23:02:45.925825 24262 ts_tablet_manager.cc:1397] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.003s
I20260927 23:02:45.926898 24265 tablet_bootstrap.cc:654] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:45.927160 24264 log.cc:824] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:45.928442 24265 log.cc:824] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:45.935469 24265 tablet_bootstrap.cc:492] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: No bootstrap required, opened a new log
I20260927 23:02:45.935863 24264 tablet_bootstrap.cc:492] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: No bootstrap required, opened a new log
I20260927 23:02:45.935897 24265 ts_tablet_manager.cc:1397] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.006s
I20260927 23:02:45.936261 24264 ts_tablet_manager.cc:1397] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.008s
I20260927 23:02:45.937825 24262 raft_consensus.cc:348] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 } } peers { permanent_uuid: "4eea257b828840cbba639591a4134b45" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37805 } } peers { permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 } }
I20260927 23:02:45.938817 24262 raft_consensus.cc:374] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:45.938953 24262 raft_consensus.cc:729] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715eb97b8a85428d81aa51b9a72d63e6, State: Initialized, Role: FOLLOWER
I20260927 23:02:45.939952 24262 consensus_queue.cc:260] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 } } peers { permanent_uuid: "4eea257b828840cbba639591a4134b45" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37805 } } peers { permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 } }
I20260927 23:02:45.942477 24262 ts_tablet_manager.cc:1428] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260927 23:02:45.946494 24265 raft_consensus.cc:348] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 } } peers { permanent_uuid: "4eea257b828840cbba639591a4134b45" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37805 } } peers { permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 } }
I20260927 23:02:45.947072 24265 raft_consensus.cc:374] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:45.947197 24265 raft_consensus.cc:729] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1c2c85134554605b9628bcc6f341daa, State: Initialized, Role: FOLLOWER
I20260927 23:02:45.948014 24265 consensus_queue.cc:260] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 } } peers { permanent_uuid: "4eea257b828840cbba639591a4134b45" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37805 } } peers { permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 } }
I20260927 23:02:45.948001 24264 raft_consensus.cc:348] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 } } peers { permanent_uuid: "4eea257b828840cbba639591a4134b45" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37805 } } peers { permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 } }
I20260927 23:02:45.948632 24264 raft_consensus.cc:374] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:45.948764 24264 raft_consensus.cc:729] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eea257b828840cbba639591a4134b45, State: Initialized, Role: FOLLOWER
I20260927 23:02:45.949519 24264 consensus_queue.cc:260] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 } } peers { permanent_uuid: "4eea257b828840cbba639591a4134b45" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37805 } } peers { permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 } }
I20260927 23:02:45.952345 24109 heartbeater.cc:503] Master 127.4.74.190:46395 was elected leader, sending a full tablet report...
I20260927 23:02:45.952368 24243 heartbeater.cc:503] Master 127.4.74.190:46395 was elected leader, sending a full tablet report...
I20260927 23:02:45.952646 24264 ts_tablet_manager.cc:1428] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260927 23:02:45.953636 24265 ts_tablet_manager.cc:1428] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: Time spent starting tablet: real 0.018s user 0.008s sys 0.009s
W20260927 23:02:46.012189 24244 tablet.cc:2369] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 23:02:46.110287 23976 tablet.cc:2369] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:46.156302 24270 raft_consensus.cc:482] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:02:46.156750 24270 raft_consensus.cc:504] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 } } peers { permanent_uuid: "4eea257b828840cbba639591a4134b45" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37805 } } peers { permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 } }
I20260927 23:02:46.158864 24270 leader_election.cc:288] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 715eb97b8a85428d81aa51b9a72d63e6 (127.4.74.129:36923), e1c2c85134554605b9628bcc6f341daa (127.4.74.130:38745)
I20260927 23:02:46.166744 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:46.159472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:51107 (local address 127.4.74.129:36923)
0927 23:02:46.159804 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:46.159815 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:46.159934 (+ 119us) server_negotiation.cc:407] Connection header received
0927 23:02:46.160237 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:46.160247 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:46.160369 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:46.160557 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:46.160990 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.162124 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.164303 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.165673 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.165829 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:46.165897 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:46.166358 (+ 461us) server_negotiation.cc:299] Negotiation successful
0927 23:02:46.166503 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260927 23:02:46.166863 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:46.159526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:36923 (local address 127.4.74.131:51107)
0927 23:02:46.159793 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:46.159822 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:46.160005 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:46.160592 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:46.160600 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:46.160638 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:46.160824 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:46.160832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.162287 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.162293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:46.164087 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:46.164102 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.165874 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.165881 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:46.166127 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:46.166177 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 23:02:46.166322 (+ 145us) client_negotiation.cc:240] Negotiation successful
0927 23:02:46.166491 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260927 23:02:46.168391 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:46.159582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:35293 (local address 127.4.74.130:38745)
0927 23:02:46.159802 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:46.159813 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:46.160978 (+ 1165us) server_negotiation.cc:407] Connection header received
0927 23:02:46.161139 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:46.161144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:46.161215 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:46.161297 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:46.161980 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.163079 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.165706 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.167169 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.167369 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:46.167454 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:46.167948 (+ 494us) server_negotiation.cc:299] Negotiation successful
0927 23:02:46.168153 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 23:02:46.168380 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4047f4f5a01d43219b89e34ac19a1756" candidate_uuid: "4eea257b828840cbba639591a4134b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "715eb97b8a85428d81aa51b9a72d63e6" is_pre_election: true
I20260927 23:02:46.169008 23930 raft_consensus.cc:2454] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eea257b828840cbba639591a4134b45 in term 0.
I20260927 23:02:46.169202 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:46.159568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:38745 (local address 127.4.74.131:35293)
0927 23:02:46.160801 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:46.160819 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:46.160977 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:46.161349 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:46.161354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:46.161381 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:46.161783 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:46.161794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.163294 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.163299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:46.165474 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:46.165492 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.167422 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.167428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:46.167674 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:46.167714 (+ 40us) client_negotiation.cc:769] Sending connection context
0927 23:02:46.168680 (+ 966us) client_negotiation.cc:240] Negotiation successful
0927 23:02:46.168848 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"mutex_wait_us":255,"thread_start_us":457,"threads_started":1}
I20260927 23:02:46.169953 24133 leader_election.cc:302] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eea257b828840cbba639591a4134b45, 715eb97b8a85428d81aa51b9a72d63e6; no voters:
I20260927 23:02:46.170472 24270 raft_consensus.cc:2781] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:02:46.170611 24270 raft_consensus.cc:482] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:02:46.170733 24270 raft_consensus.cc:3037] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:46.170589 24064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4047f4f5a01d43219b89e34ac19a1756" candidate_uuid: "4eea257b828840cbba639591a4134b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c2c85134554605b9628bcc6f341daa" is_pre_election: true
I20260927 23:02:46.171196 24064 raft_consensus.cc:2454] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eea257b828840cbba639591a4134b45 in term 0.
I20260927 23:02:46.174014 24270 raft_consensus.cc:504] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 } } peers { permanent_uuid: "4eea257b828840cbba639591a4134b45" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37805 } } peers { permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 } }
I20260927 23:02:46.174885 24270 leader_election.cc:288] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [CANDIDATE]: Term 1 election: Requested vote from peers 715eb97b8a85428d81aa51b9a72d63e6 (127.4.74.129:36923), e1c2c85134554605b9628bcc6f341daa (127.4.74.130:38745)
I20260927 23:02:46.175782 23930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4047f4f5a01d43219b89e34ac19a1756" candidate_uuid: "4eea257b828840cbba639591a4134b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "715eb97b8a85428d81aa51b9a72d63e6"
I20260927 23:02:46.175966 24064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4047f4f5a01d43219b89e34ac19a1756" candidate_uuid: "4eea257b828840cbba639591a4134b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1c2c85134554605b9628bcc6f341daa"
I20260927 23:02:46.176070 23930 raft_consensus.cc:3037] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:46.176272 24064 raft_consensus.cc:3037] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:46.179050 24064 raft_consensus.cc:2454] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eea257b828840cbba639591a4134b45 in term 1.
I20260927 23:02:46.179083 23930 raft_consensus.cc:2454] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eea257b828840cbba639591a4134b45 in term 1.
I20260927 23:02:46.179780 24131 leader_election.cc:302] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eea257b828840cbba639591a4134b45, e1c2c85134554605b9628bcc6f341daa; no voters:
I20260927 23:02:46.180234 24270 raft_consensus.cc:2781] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:46.180807 24270 raft_consensus.cc:686] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 1 LEADER]: Becoming Leader. State: Replica: 4eea257b828840cbba639591a4134b45, State: Running, Role: LEADER
I20260927 23:02:46.181528 24270 consensus_queue.cc:237] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 } } peers { permanent_uuid: "4eea257b828840cbba639591a4134b45" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37805 } } peers { permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 } }
W20260927 23:02:46.185345 24110 tablet.cc:2369] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:46.189581 23785 catalog_manager.cc:5495] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4eea257b828840cbba639591a4134b45 (127.4.74.131). New cstate: current_term: 1 leader_uuid: "4eea257b828840cbba639591a4134b45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4eea257b828840cbba639591a4134b45" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 37805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 } health_report { overall_health: UNKNOWN } } }
I20260927 23:02:46.212215 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:46.204620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.4.74.131:37805)
0927 23:02:46.204840 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:46.204845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:46.205111 (+ 266us) server_negotiation.cc:407] Connection header received
0927 23:02:46.205312 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:46.205316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:46.205413 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:46.205596 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:02:46.206372 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.207189 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.209282 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.209815 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.209960 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:46.210829 (+ 869us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:02:46.211426 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:02:46.211668 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:46.211872 (+ 204us) server_negotiation.cc:299] Negotiation successful
0927 23:02:46.211973 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 23:02:46.223799 24064 raft_consensus.cc:1261] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa [term 1 FOLLOWER]: Refusing update from remote peer 4eea257b828840cbba639591a4134b45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:46.223918 23930 raft_consensus.cc:1261] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6 [term 1 FOLLOWER]: Refusing update from remote peer 4eea257b828840cbba639591a4134b45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:46.225270 24272 consensus_queue.cc:1035] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "715eb97b8a85428d81aa51b9a72d63e6" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 36923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:46.225776 24270 consensus_queue.cc:1035] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1c2c85134554605b9628bcc6f341daa" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 38745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:46.245589 24277 mvcc.cc:205] Tried to move back new op lower bound from 7334093480838836224 to 7334093480686288896. Current Snapshot: MvccSnapshot[applied={T|T < 7334093480838836224}]
I20260927 23:02:46.266019 23785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:46.266342 23785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:46.271073 23785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0522dfc58603445abfddbe41ccdcda74: Sending DeleteTablet for 3 replicas of tablet 4047f4f5a01d43219b89e34ac19a1756
I20260927 23:02:46.272538 23910 tablet_service.cc:1514] Processing DeleteTablet for tablet 4047f4f5a01d43219b89e34ac19a1756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:46 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:48646
I20260927 23:02:46.272661 24178 tablet_service.cc:1514] Processing DeleteTablet for tablet 4047f4f5a01d43219b89e34ac19a1756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:46 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:41092
I20260927 23:02:46.272971 24044 tablet_service.cc:1514] Processing DeleteTablet for tablet 4047f4f5a01d43219b89e34ac19a1756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:46 UTC) from {username='kudu', principal='kudu/127.4.74.190@KRBTEST.COM'} at 127.0.0.1:42420
I20260927 23:02:46.277221 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 23847
I20260927 23:02:46.294576 24287 tablet_replica.cc:330] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: stopping tablet replica
I20260927 23:02:46.295050 24285 tablet_replica.cc:330] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: stopping tablet replica
I20260927 23:02:46.295825 24285 raft_consensus.cc:2229] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:02:46.296633 24287 raft_consensus.cc:2229] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:46.296766 24285 raft_consensus.cc:2258] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:46.297976 24287 raft_consensus.cc:2258] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:46.298058 24286 tablet_replica.cc:330] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: stopping tablet replica
I20260927 23:02:46.298930 24286 raft_consensus.cc:2229] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:46.299677 24286 raft_consensus.cc:2258] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:46.299801 24285 ts_tablet_manager.cc:1905] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:46.300618 24287 ts_tablet_manager.cc:1905] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:46.302631 24286 ts_tablet_manager.cc:1905] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:46.309264 24287 ts_tablet_manager.cc:1918] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:46.309599 24287 log.cc:1196] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/4047f4f5a01d43219b89e34ac19a1756
I20260927 23:02:46.310351 24287 ts_tablet_manager.cc:1939] T 4047f4f5a01d43219b89e34ac19a1756 P e1c2c85134554605b9628bcc6f341daa: Deleting consensus metadata
I20260927 23:02:46.311534 24286 ts_tablet_manager.cc:1918] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:46.311810 24286 log.cc:1196] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/4047f4f5a01d43219b89e34ac19a1756
I20260927 23:02:46.312455 24286 ts_tablet_manager.cc:1939] T 4047f4f5a01d43219b89e34ac19a1756 P 715eb97b8a85428d81aa51b9a72d63e6: Deleting consensus metadata
I20260927 23:02:46.313033 23770 catalog_manager.cc:4841] TS e1c2c85134554605b9628bcc6f341daa (127.4.74.130:38745): tablet 4047f4f5a01d43219b89e34ac19a1756 (table test-table [id=3704406ff9f941e5bd0aa3f8f77af402]) successfully deleted
I20260927 23:02:46.313511 24285 ts_tablet_manager.cc:1918] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:46.313733 24285 log.cc:1196] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/4047f4f5a01d43219b89e34ac19a1756
I20260927 23:02:46.314446 24285 ts_tablet_manager.cc:1939] T 4047f4f5a01d43219b89e34ac19a1756 P 4eea257b828840cbba639591a4134b45: Deleting consensus metadata
I20260927 23:02:46.314992 23772 catalog_manager.cc:4841] TS 715eb97b8a85428d81aa51b9a72d63e6 (127.4.74.129:36923): tablet 4047f4f5a01d43219b89e34ac19a1756 (table test-table [id=3704406ff9f941e5bd0aa3f8f77af402]) successfully deleted
I20260927 23:02:46.317227 23770 catalog_manager.cc:4841] TS 4eea257b828840cbba639591a4134b45 (127.4.74.131:37805): tablet 4047f4f5a01d43219b89e34ac19a1756 (table test-table [id=3704406ff9f941e5bd0aa3f8f77af402]) successfully deleted
Sep 27 23:02:46 dist-test-slave-5pd3 krb5kdc[23741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.129@KRBTEST.COM
I20260927 23:02:46.331938 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:46.281227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48648 (local address 127.4.74.129:36923)
0927 23:02:46.281499 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:46.281510 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:46.294640 (+ 13130us) server_negotiation.cc:407] Connection header received
0927 23:02:46.295449 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:46.295461 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:46.295664 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:46.295758 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:46.300879 (+ 5121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.302192 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.313840 (+ 11648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.314806 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.314985 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:46.321671 (+ 6686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:46.321701 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:46.321727 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:46.321798 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:46.326802 (+ 5004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:46.328654 (+ 1852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:46.328668 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:46.328674 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:46.328936 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:46.329529 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:46.329535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:46.329538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:46.330193 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:46.330627 (+ 434us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:46.331061 (+ 434us) server_negotiation.cc:299] Negotiation successful
0927 23:02:46.331387 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260927 23:02:46.332964 23970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:46.429418 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 23847
I20260927 23:02:46.450316 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 23981
Sep 27 23:02:46 dist-test-slave-5pd3 krb5kdc[23741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.130@KRBTEST.COM
I20260927 23:02:46.473774 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:46.451464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42426 (local address 127.4.74.130:38745)
0927 23:02:46.451815 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:46.451821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:46.451843 (+ 22us) server_negotiation.cc:407] Connection header received
0927 23:02:46.452005 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:46.452009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:46.452095 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:46.452217 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:46.454363 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.455180 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.457368 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.457819 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.457955 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:46.464649 (+ 6694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:46.464679 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:46.464706 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:46.464778 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:46.469761 (+ 4983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:46.470859 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:46.470870 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:46.470875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:46.471088 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:46.471707 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:46.471713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:46.471716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:46.472317 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:46.472698 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:46.473007 (+ 309us) server_negotiation.cc:299] Negotiation successful
0927 23:02:46.473300 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222}
I20260927 23:02:46.474779 24104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:46.568378 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 23981
I20260927 23:02:46.594578 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 24115
Sep 27 23:02:46 dist-test-slave-5pd3 krb5kdc[23741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.131@KRBTEST.COM
I20260927 23:02:46.615888 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:46.595766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.4.74.131:37805)
0927 23:02:46.596114 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:46.596120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:46.596145 (+ 25us) server_negotiation.cc:407] Connection header received
0927 23:02:46.596227 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:46.596234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:46.596313 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:46.596438 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:46.597802 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.598722 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.600685 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:46.601313 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:46.601489 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:46.607317 (+ 5828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:46.607336 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:46.607358 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:46.607415 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:46.611378 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:46.612877 (+ 1499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:46.612885 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:46.612887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:46.613084 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:46.613687 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:46.613699 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:46.613701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:46.614144 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:46.614419 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:46.615010 (+ 591us) server_negotiation.cc:299] Negotiation successful
0927 23:02:46.615360 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260927 23:02:46.616739 24238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:46.761152 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 24115
I20260927 23:02:46.786147 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 23754
I20260927 23:02:46.787482 23816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:46.921218 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 23754
2026-09-27T23:02:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5130 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-27T23:02:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:02:46Z Disabled control of system clock
I20260927 23:02:47.016599 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:42019
--webserver_interface=127.4.74.190
--webserver_port=0
--builtin_ntp_servers=127.4.74.148:46041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:42019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 23:02:47.489591 24296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:47.490041 24296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:47.490236 24296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:47.499132 24296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:47.499315 24296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:47.499358 24296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:47.499418 24296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:47.499531 24296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:47.510728 24296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:42019
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:42019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:47.512720 24296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:47.515260 24296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:47.531117 24301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:47.531059 24302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:47.531430 24304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:47.532397 24296 server_base.cc:1034] running on GCE node
I20260927 23:02:47.533524 24296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:47.535686 24296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:47.536957 24296 hybrid_clock.cc:648] HybridClock initialized: now 1790550167536882 us; error 85 us; skew 500 ppm
I20260927 23:02:47.537433 24296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:47.540635 24296 webserver.cc:460] Webserver started at http://127.4.74.190:42923/ using document root <none> and password file <none>
I20260927 23:02:47.541607 24296 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:47.541769 24296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:47.542182 24296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:47.546056 24296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "6b8d41618d564ecf87e6c640609e373a"
format_stamp: "Formatted at 2026-09-27 23:02:47 on dist-test-slave-5pd3"
server_key: "8407094b4312b0185ce598df32c2c939"
server_key_iv: "a6d4200783fe7bd4f6e3032204bbacb2"
server_key_version: "encryptionkey@0"
I20260927 23:02:47.547171 24296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "6b8d41618d564ecf87e6c640609e373a"
format_stamp: "Formatted at 2026-09-27 23:02:47 on dist-test-slave-5pd3"
server_key: "8407094b4312b0185ce598df32c2c939"
server_key_iv: "a6d4200783fe7bd4f6e3032204bbacb2"
server_key_version: "encryptionkey@0"
I20260927 23:02:47.554445 24296 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.004s
I20260927 23:02:47.559333 24310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:47.561314 24296 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:02:47.561592 24296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "6b8d41618d564ecf87e6c640609e373a"
format_stamp: "Formatted at 2026-09-27 23:02:47 on dist-test-slave-5pd3"
server_key: "8407094b4312b0185ce598df32c2c939"
server_key_iv: "a6d4200783fe7bd4f6e3032204bbacb2"
server_key_version: "encryptionkey@0"
I20260927 23:02:47.561899 24296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:47.576599 24296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:47.577709 24296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:47.578044 24296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:47.600867 24296 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:42019
I20260927 23:02:47.600893 24361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:42019 every 8 connection(s)
I20260927 23:02:47.602824 24296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:47.605026 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 24296
I20260927 23:02:47.605263 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:02:47.605793 4394 external_mini_cluster.cc:1428] Setting key ae2d236169389a3276cfb2f518e8e313
I20260927 23:02:47.608429 24362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:47.623950 24365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:47.608386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.4.74.190:42019)
0927 23:02:47.609788 (+ 1402us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:47.609806 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:47.609870 (+ 64us) server_negotiation.cc:407] Connection header received
0927 23:02:47.612140 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:47.612178 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:47.612892 (+ 714us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:47.613420 (+ 528us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:47.614096 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:47.616248 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:47.617907 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:47.618959 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:47.619187 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:47.620739 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:47.620749 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:47.620770 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:47.621615 (+ 845us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:47.621718 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:47.621751 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:47.621899 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:47.622247 (+ 348us) server_negotiation.cc:299] Negotiation successful
0927 23:02:47.622602 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":442,"threads_started":1}
I20260927 23:02:47.624501 24362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a: Bootstrap starting.
I20260927 23:02:47.628314 24362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:47.629307 24362 log.cc:824] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:47.632716 24362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a: No bootstrap required, opened a new log
I20260927 23:02:47.640390 24362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8d41618d564ecf87e6c640609e373a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42019 } }
I20260927 23:02:47.640916 24362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:47.641026 24362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b8d41618d564ecf87e6c640609e373a, State: Initialized, Role: FOLLOWER
I20260927 23:02:47.641630 24362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8d41618d564ecf87e6c640609e373a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42019 } }
I20260927 23:02:47.641899 24362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:47.642056 24362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:47.642246 24362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:47.644555 24362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8d41618d564ecf87e6c640609e373a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42019 } }
I20260927 23:02:47.645121 24362 leader_election.cc:302] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b8d41618d564ecf87e6c640609e373a; no voters:
I20260927 23:02:47.645819 24362 leader_election.cc:288] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:02:47.646117 24367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:47.647113 24367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [term 1 LEADER]: Becoming Leader. State: Replica: 6b8d41618d564ecf87e6c640609e373a, State: Running, Role: LEADER
I20260927 23:02:47.647883 24367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8d41618d564ecf87e6c640609e373a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42019 } }
I20260927 23:02:47.648216 24362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:47.652408 24368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b8d41618d564ecf87e6c640609e373a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8d41618d564ecf87e6c640609e373a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42019 } } }
I20260927 23:02:47.652956 24368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:47.653569 24369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b8d41618d564ecf87e6c640609e373a. Latest consensus state: current_term: 1 leader_uuid: "6b8d41618d564ecf87e6c640609e373a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8d41618d564ecf87e6c640609e373a" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 42019 } } }
I20260927 23:02:47.653906 24369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:47.654048 24378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:47.661185 24378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:47.672992 24378 catalog_manager.cc:1347] Generated new cluster ID: f4134492877b4a56ae09a712687a913f
I20260927 23:02:47.673225 24378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:47.722074 24378 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:02:47.724318 24378 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:47.737178 24378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a: Generated new TSK 0
I20260927 23:02:47.738340 24378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:02:47.751523 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.129:0
--local_ip_for_outbound_sockets=127.4.74.129
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:42019
--builtin_ntp_servers=127.4.74.148:46041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 23:02:48.158751 24386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:48.159058 24386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:48.159147 24386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:48.168211 24386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:48.168341 24386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:48.168455 24386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.129
I20260927 23:02:48.178320 24386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.74.129
--webserver_port=0
--tserver_master_addrs=127.4.74.190:42019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.129
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:48.179786 24386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:48.181699 24386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:48.199302 24392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:48.199322 24391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:48.200120 24394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:48.200706 24386 server_base.cc:1034] running on GCE node
I20260927 23:02:48.201675 24386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:48.203069 24386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:48.204443 24386 hybrid_clock.cc:648] HybridClock initialized: now 1790550168204319 us; error 118 us; skew 500 ppm
I20260927 23:02:48.204957 24386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:48.209548 24386 webserver.cc:460] Webserver started at http://127.4.74.129:38821/ using document root <none> and password file <none>
I20260927 23:02:48.210734 24386 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:48.210938 24386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:48.211417 24386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:48.216578 24386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/data/instance:
uuid: "efe597dd90ac45fbbbac2ca97b77aafa"
format_stamp: "Formatted at 2026-09-27 23:02:48 on dist-test-slave-5pd3"
server_key: "50f509b86a6b0f6a0b086b6bd7e5d8d1"
server_key_iv: "b58cc62d7231975e8f98c09193f1f948"
server_key_version: "encryptionkey@0"
I20260927 23:02:48.218418 24386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance:
uuid: "efe597dd90ac45fbbbac2ca97b77aafa"
format_stamp: "Formatted at 2026-09-27 23:02:48 on dist-test-slave-5pd3"
server_key: "50f509b86a6b0f6a0b086b6bd7e5d8d1"
server_key_iv: "b58cc62d7231975e8f98c09193f1f948"
server_key_version: "encryptionkey@0"
I20260927 23:02:48.226348 24386 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260927 23:02:48.232000 24400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:48.234027 24386 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:02:48.234350 24386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/wal
uuid: "efe597dd90ac45fbbbac2ca97b77aafa"
format_stamp: "Formatted at 2026-09-27 23:02:48 on dist-test-slave-5pd3"
server_key: "50f509b86a6b0f6a0b086b6bd7e5d8d1"
server_key_iv: "b58cc62d7231975e8f98c09193f1f948"
server_key_version: "encryptionkey@0"
I20260927 23:02:48.234634 24386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:48.266278 24386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:48.267465 24386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:48.267915 24386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:48.269197 24386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:48.271220 24386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:48.271323 24386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:48.271478 24386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:48.271612 24386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260927 23:02:48.314704 24386 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.129:35431
I20260927 23:02:48.314774 24512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.129:35431 every 8 connection(s)
I20260927 23:02:48.316555 24386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/data/info.pb
I20260927 23:02:48.327099 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 24386
I20260927 23:02:48.327512 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/wal/instance
I20260927 23:02:48.328265 4394 external_mini_cluster.cc:1428] Setting key 7adf23924041254021224141fdcff2fb
I20260927 23:02:48.331717 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.130:0
--local_ip_for_outbound_sockets=127.4.74.130
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:42019
--builtin_ntp_servers=127.4.74.148:46041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 23:02:48.335028 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:48.319883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.129:49615 (local address 127.4.74.190:42019)
0927 23:02:48.320492 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:48.320498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:48.322123 (+ 1625us) server_negotiation.cc:407] Connection header received
0927 23:02:48.324836 (+ 2713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:48.324842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:48.324976 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:48.325174 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:48.326975 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:48.328387 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:48.330819 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:48.331464 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:48.331655 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:48.333855 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:48.333864 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:48.333873 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:48.333903 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:48.333964 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:48.333984 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:48.334097 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:48.334432 (+ 335us) server_negotiation.cc:299] Negotiation successful
0927 23:02:48.334557 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":306,"threads_started":1}
I20260927 23:02:48.336427 24516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:48.320473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:42019 (local address 127.4.74.129:49615)
0927 23:02:48.321863 (+ 1390us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:48.321941 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:48.324376 (+ 2435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:48.325553 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:48.325565 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:48.325715 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:48.326557 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:48.326580 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:48.328667 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:48.328672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:48.330612 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:48.330633 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:48.331832 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:48.331839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:48.332202 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:48.333335 (+ 1133us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 23:02:48.333339 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:48.333423 (+ 84us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 23:02:48.333565 (+ 142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:48.334150 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:48.334161 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:48.334169 (+ 8us) client_negotiation.cc:769] Sending connection context
0927 23:02:48.334382 (+ 213us) client_negotiation.cc:240] Negotiation successful
0927 23:02:48.334867 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":438,"threads_started":1}
I20260927 23:02:48.338634 24513 heartbeater.cc:347] Connected to a master server at 127.4.74.190:42019
I20260927 23:02:48.339077 24513 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:48.340268 24513 heartbeater.cc:511] Master 127.4.74.190:42019 requested a full tablet report, sending...
I20260927 23:02:48.343358 24327 ts_manager.cc:194] Registered new tserver with Master: efe597dd90ac45fbbbac2ca97b77aafa (127.4.74.129:35431)
I20260927 23:02:48.345283 24327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.129:49615
W20260927 23:02:48.686683 24517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:48.687006 24517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:48.687145 24517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:48.694619 24517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:48.694753 24517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:48.694954 24517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.130
I20260927 23:02:48.704453 24517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.74.130
--webserver_port=0
--tserver_master_addrs=127.4.74.190:42019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.130
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:48.705991 24517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:48.707847 24517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:48.722977 24522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:48.722981 24525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:48.722980 24523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:48.724252 24517 server_base.cc:1034] running on GCE node
I20260927 23:02:48.725000 24517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:48.726153 24517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:48.727411 24517 hybrid_clock.cc:648] HybridClock initialized: now 1790550168727328 us; error 93 us; skew 500 ppm
I20260927 23:02:48.727869 24517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:48.730466 24517 webserver.cc:460] Webserver started at http://127.4.74.130:39529/ using document root <none> and password file <none>
I20260927 23:02:48.731220 24517 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:48.731359 24517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:48.731772 24517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:48.734790 24517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/data/instance:
uuid: "6e810acae7564d82a8b0436936759777"
format_stamp: "Formatted at 2026-09-27 23:02:48 on dist-test-slave-5pd3"
server_key: "be53168a8293cb2a568e77bce0aae1a6"
server_key_iv: "27573cb8c14422e40c6c0c0aab69024d"
server_key_version: "encryptionkey@0"
I20260927 23:02:48.735692 24517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance:
uuid: "6e810acae7564d82a8b0436936759777"
format_stamp: "Formatted at 2026-09-27 23:02:48 on dist-test-slave-5pd3"
server_key: "be53168a8293cb2a568e77bce0aae1a6"
server_key_iv: "27573cb8c14422e40c6c0c0aab69024d"
server_key_version: "encryptionkey@0"
I20260927 23:02:48.741868 24517 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260927 23:02:48.746417 24531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:48.748565 24517 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260927 23:02:48.748806 24517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/wal
uuid: "6e810acae7564d82a8b0436936759777"
format_stamp: "Formatted at 2026-09-27 23:02:48 on dist-test-slave-5pd3"
server_key: "be53168a8293cb2a568e77bce0aae1a6"
server_key_iv: "27573cb8c14422e40c6c0c0aab69024d"
server_key_version: "encryptionkey@0"
I20260927 23:02:48.749128 24517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:48.766249 24517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:48.767205 24517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:48.767517 24517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:48.768795 24517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:48.770545 24517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:48.770669 24517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:48.770828 24517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:48.770916 24517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:48.811378 24517 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.130:43337
I20260927 23:02:48.811486 24643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.130:43337 every 8 connection(s)
I20260927 23:02:48.812960 24517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/data/info.pb
I20260927 23:02:48.813298 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 24517
I20260927 23:02:48.813491 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/wal/instance
I20260927 23:02:48.814152 4394 external_mini_cluster.cc:1428] Setting key 94793ca0a8b9e1007ca45d96ca80cb8c
I20260927 23:02:48.817873 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.74.131:0
--local_ip_for_outbound_sockets=127.4.74.131
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:42019
--builtin_ntp_servers=127.4.74.148:46041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 23:02:48.837844 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:48.816781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:35177 (local address 127.4.74.190:42019)
0927 23:02:48.816995 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:48.817002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:48.819220 (+ 2218us) server_negotiation.cc:407] Connection header received
0927 23:02:48.822751 (+ 3531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:48.822757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:48.822843 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:48.822980 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:48.824999 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:48.826327 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:48.828309 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:48.828729 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:48.828888 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:48.836538 (+ 7650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:48.836550 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:48.836558 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:48.836594 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:48.836653 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:48.836675 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:48.836808 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:48.837276 (+ 468us) server_negotiation.cc:299] Negotiation successful
0927 23:02:48.837451 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 23:02:48.839280 24646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:48.817448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:42019 (local address 127.4.74.130:35177)
0927 23:02:48.818952 (+ 1504us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:48.819029 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:48.822257 (+ 3228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:48.823372 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:48.823392 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:48.823600 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:48.824723 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:48.824760 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:48.826623 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:48.826629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:48.828119 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:48.828137 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:48.828993 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:48.829001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:48.829244 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:48.835848 (+ 6604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 23:02:48.835853 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:48.835965 (+ 112us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 23:02:48.836122 (+ 157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:48.836853 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:48.836864 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:48.836874 (+ 10us) client_negotiation.cc:769] Sending connection context
0927 23:02:48.837206 (+ 332us) client_negotiation.cc:240] Negotiation successful
0927 23:02:48.837839 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":383,"threads_started":1}
I20260927 23:02:48.841710 24644 heartbeater.cc:347] Connected to a master server at 127.4.74.190:42019
I20260927 23:02:48.842167 24644 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:48.843256 24644 heartbeater.cc:511] Master 127.4.74.190:42019 requested a full tablet report, sending...
I20260927 23:02:48.845494 24327 ts_manager.cc:194] Registered new tserver with Master: 6e810acae7564d82a8b0436936759777 (127.4.74.130:43337)
I20260927 23:02:48.846577 24327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.130:35177
W20260927 23:02:49.182828 24647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:49.183140 24647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:49.183276 24647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:49.191089 24647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:49.191220 24647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:49.191409 24647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.74.131
I20260927 23:02:49.201339 24647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:46041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.74.131
--webserver_port=0
--tserver_master_addrs=127.4.74.190:42019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.74.131
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:49.202729 24647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:49.204690 24647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:49.221647 24652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:49.221695 24655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:49.221999 24653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:49.222921 24647 server_base.cc:1034] running on GCE node
I20260927 23:02:49.223809 24647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:49.224948 24647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:49.226263 24647 hybrid_clock.cc:648] HybridClock initialized: now 1790550169226176 us; error 81 us; skew 500 ppm
I20260927 23:02:49.226661 24647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:02:49.229566 24647 webserver.cc:460] Webserver started at http://127.4.74.131:35483/ using document root <none> and password file <none>
I20260927 23:02:49.230350 24647 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:49.230489 24647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:49.230836 24647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:49.234500 24647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/data/instance:
uuid: "af67e8ccc1ee410f8e817aea0514abb9"
format_stamp: "Formatted at 2026-09-27 23:02:49 on dist-test-slave-5pd3"
server_key: "d712084da77dafc7589b595c72bcc5cc"
server_key_iv: "c56d30c5d45fc1523c11754cb25780ac"
server_key_version: "encryptionkey@0"
I20260927 23:02:49.235734 24647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance:
uuid: "af67e8ccc1ee410f8e817aea0514abb9"
format_stamp: "Formatted at 2026-09-27 23:02:49 on dist-test-slave-5pd3"
server_key: "d712084da77dafc7589b595c72bcc5cc"
server_key_iv: "c56d30c5d45fc1523c11754cb25780ac"
server_key_version: "encryptionkey@0"
I20260927 23:02:49.242172 24647 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:02:49.247231 24661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:49.249564 24647 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260927 23:02:49.249814 24647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/wal
uuid: "af67e8ccc1ee410f8e817aea0514abb9"
format_stamp: "Formatted at 2026-09-27 23:02:49 on dist-test-slave-5pd3"
server_key: "d712084da77dafc7589b595c72bcc5cc"
server_key_iv: "c56d30c5d45fc1523c11754cb25780ac"
server_key_version: "encryptionkey@0"
I20260927 23:02:49.250123 24647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:49.268306 24647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:49.269225 24647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:49.269542 24647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:49.270696 24647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:02:49.272503 24647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:02:49.272637 24647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:49.272789 24647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:02:49.272886 24647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:49.313083 24647 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.131:41261
I20260927 23:02:49.313215 24773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.131:41261 every 8 connection(s)
I20260927 23:02:49.314770 24647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/data/info.pb
I20260927 23:02:49.324872 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 24647
I20260927 23:02:49.325155 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/wal/instance
I20260927 23:02:49.325719 4394 external_mini_cluster.cc:1428] Setting key fd3822678d5785ed72b173765896efe6
I20260927 23:02:49.334084 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.318459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.131:57003 (local address 127.4.74.190:42019)
0927 23:02:49.318732 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.318739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.322864 (+ 4125us) server_negotiation.cc:407] Connection header received
0927 23:02:49.324884 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.324892 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.325023 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.325210 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:49.326690 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.327711 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.330526 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.331039 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.331184 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.333181 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:49.333187 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:49.333190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:49.333215 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:49.333263 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:49.333277 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:49.333379 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.333651 (+ 272us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.333757 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260927 23:02:49.335956 24776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.319321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:42019 (local address 127.4.74.131:57003)
0927 23:02:49.322639 (+ 3318us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.322690 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:49.324485 (+ 1795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:49.325569 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:49.325578 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:49.325693 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:02:49.326384 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.326401 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.327972 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.327989 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.330296 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.330323 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.331398 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.331405 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.331796 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.332692 (+ 896us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 23:02:49.332695 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:02:49.332753 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 23:02:49.332873 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:02:49.333402 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:02:49.333412 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:02:49.333419 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 23:02:49.333623 (+ 204us) client_negotiation.cc:240] Negotiation successful
0927 23:02:49.334391 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2768,"thread_start_us":2520,"threads_started":1}
I20260927 23:02:49.338093 24774 heartbeater.cc:347] Connected to a master server at 127.4.74.190:42019
I20260927 23:02:49.338578 24774 heartbeater.cc:464] Registering TS with master...
I20260927 23:02:49.339738 24774 heartbeater.cc:511] Master 127.4.74.190:42019 requested a full tablet report, sending...
I20260927 23:02:49.342010 24326 ts_manager.cc:194] Registered new tserver with Master: af67e8ccc1ee410f8e817aea0514abb9 (127.4.74.131:41261)
I20260927 23:02:49.343381 4394 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:02:49.343326 24326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.74.131:57003
I20260927 23:02:49.349926 24513 heartbeater.cc:503] Master 127.4.74.190:42019 was elected leader, sending a full tablet report...
I20260927 23:02:49.357115 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.349231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.4.74.190:42019)
0927 23:02:49.349572 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.349584 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.350127 (+ 543us) server_negotiation.cc:407] Connection header received
0927 23:02:49.350372 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.350378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.350467 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.350599 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:49.351142 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.352439 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.354094 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.354663 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.354851 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.356232 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:49.356241 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:49.356250 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:49.356288 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:49.356333 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:49.356348 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:49.356437 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.356622 (+ 185us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.356725 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260927 23:02:49.362658 24326 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:02:49.372125 24326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:02:49.403211 24788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.387434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:35431 (local address 127.0.0.1:54618)
0927 23:02:49.389557 (+ 2123us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.389609 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:49.389860 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:49.391028 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:49.391036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:49.391069 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:49.391586 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.391608 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.394255 (+ 2647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.394265 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.397996 (+ 3731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.398032 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.401620 (+ 3588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.401633 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.402058 (+ 425us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.402151 (+ 93us) client_negotiation.cc:769] Sending connection context
0927 23:02:49.402362 (+ 211us) client_negotiation.cc:240] Negotiation successful
0927 23:02:49.402671 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1740,"thread_start_us":1856,"threads_started":1}
I20260927 23:02:49.404731 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.387901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54618 (local address 127.4.74.129:35431)
0927 23:02:49.388661 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.388671 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.389795 (+ 1124us) server_negotiation.cc:407] Connection header received
0927 23:02:49.390166 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.390178 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.390469 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.390691 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:49.391880 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.394012 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.398453 (+ 4441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.400748 (+ 2295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.400948 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.401388 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.403884 (+ 2496us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.404190 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":326,"threads_started":1}
I20260927 23:02:49.409130 24789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.389096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.130:43337 (local address 127.0.0.1:54600)
0927 23:02:49.389838 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.389860 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:49.390102 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:49.391650 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:49.391658 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:49.391706 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:49.392407 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.392422 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.395386 (+ 2964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.395396 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.402911 (+ 7515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.402944 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.408179 (+ 5235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.408189 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.408430 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.408473 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 23:02:49.408582 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 23:02:49.408736 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":873,"threads_started":1}
I20260927 23:02:49.410107 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.387988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.4.74.130:43337)
0927 23:02:49.388825 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.388836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.389981 (+ 1145us) server_negotiation.cc:407] Connection header received
0927 23:02:49.390310 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.390318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.390655 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.390908 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:49.392651 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.395068 (+ 2417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.403319 (+ 8251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.406030 (+ 2711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.406240 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.406660 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.409479 (+ 2819us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.409730 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":689,"threads_started":1}
I20260927 23:02:49.410511 24448 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2ed3020e98495291c2f4659fc21f40 (DEFAULT_TABLE table=test-table [id=fdfcbc8e1efc46eebee9b5d73d50ba77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:49.412153 24448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2ed3020e98495291c2f4659fc21f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:49.414505 24579 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2ed3020e98495291c2f4659fc21f40 (DEFAULT_TABLE table=test-table [id=fdfcbc8e1efc46eebee9b5d73d50ba77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:49.415998 24579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2ed3020e98495291c2f4659fc21f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:49.416971 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.390728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:41261 (local address 127.0.0.1:39140)
0927 23:02:49.406584 (+ 15856us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.406611 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:49.406812 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:49.408991 (+ 2179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:49.409001 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:49.409033 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:49.409557 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.409571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.411705 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.411710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.413492 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.413513 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.415979 (+ 2466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.415985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.416205 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.416249 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 23:02:49.416389 (+ 140us) client_negotiation.cc:240] Negotiation successful
0927 23:02:49.416570 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15677,"thread_start_us":16613,"threads_started":1}
I20260927 23:02:49.417286 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.391075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39140 (local address 127.4.74.131:41261)
0927 23:02:49.408054 (+ 16979us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.408067 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.408115 (+ 48us) server_negotiation.cc:407] Connection header received
0927 23:02:49.408263 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.408273 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.408748 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.409022 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:49.409773 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.411457 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.413766 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.415319 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.415472 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.415835 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.416460 (+ 625us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.416661 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16728,"thread_start_us":16506,"threads_started":1}
I20260927 23:02:49.421528 24709 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2ed3020e98495291c2f4659fc21f40 (DEFAULT_TABLE table=test-table [id=fdfcbc8e1efc46eebee9b5d73d50ba77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:02:49.423843 24709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2ed3020e98495291c2f4659fc21f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:49.428565 24792 tablet_bootstrap.cc:492] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: Bootstrap starting.
I20260927 23:02:49.431182 24793 tablet_bootstrap.cc:492] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: Bootstrap starting.
I20260927 23:02:49.434448 24792 tablet_bootstrap.cc:654] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:49.434856 24793 tablet_bootstrap.cc:654] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:49.435952 24793 log.cc:824] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:49.436098 24792 log.cc:824] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:49.439291 24792 tablet_bootstrap.cc:492] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: No bootstrap required, opened a new log
I20260927 23:02:49.439692 24792 ts_tablet_manager.cc:1397] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.003s
I20260927 23:02:49.439872 24793 tablet_bootstrap.cc:492] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: No bootstrap required, opened a new log
I20260927 23:02:49.440231 24793 ts_tablet_manager.cc:1397] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260927 23:02:49.442070 24796 tablet_bootstrap.cc:492] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: Bootstrap starting.
I20260927 23:02:49.446138 24796 tablet_bootstrap.cc:654] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:49.447464 24796 log.cc:824] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:49.450596 24793 raft_consensus.cc:348] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 } } peers { permanent_uuid: "6e810acae7564d82a8b0436936759777" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43337 } } peers { permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 } }
I20260927 23:02:49.451081 24793 raft_consensus.cc:374] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:49.451179 24793 raft_consensus.cc:729] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e810acae7564d82a8b0436936759777, State: Initialized, Role: FOLLOWER
I20260927 23:02:49.450944 24792 raft_consensus.cc:348] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 } } peers { permanent_uuid: "6e810acae7564d82a8b0436936759777" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43337 } } peers { permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 } }
I20260927 23:02:49.451409 24792 raft_consensus.cc:374] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:49.451438 24796 tablet_bootstrap.cc:492] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: No bootstrap required, opened a new log
I20260927 23:02:49.451577 24792 raft_consensus.cc:729] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe597dd90ac45fbbbac2ca97b77aafa, State: Initialized, Role: FOLLOWER
I20260927 23:02:49.451823 24796 ts_tablet_manager.cc:1397] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260927 23:02:49.452278 24793 consensus_queue.cc:260] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 } } peers { permanent_uuid: "6e810acae7564d82a8b0436936759777" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43337 } } peers { permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 } }
I20260927 23:02:49.452283 24792 consensus_queue.cc:260] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 } } peers { permanent_uuid: "6e810acae7564d82a8b0436936759777" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43337 } } peers { permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 } }
I20260927 23:02:49.454186 24792 ts_tablet_manager.cc:1428] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260927 23:02:49.456451 24644 heartbeater.cc:503] Master 127.4.74.190:42019 was elected leader, sending a full tablet report...
I20260927 23:02:49.461074 24796 raft_consensus.cc:348] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 } } peers { permanent_uuid: "6e810acae7564d82a8b0436936759777" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43337 } } peers { permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 } }
I20260927 23:02:49.461547 24796 raft_consensus.cc:374] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:49.461656 24796 raft_consensus.cc:729] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af67e8ccc1ee410f8e817aea0514abb9, State: Initialized, Role: FOLLOWER
I20260927 23:02:49.461766 24793 ts_tablet_manager.cc:1428] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: Time spent starting tablet: real 0.021s user 0.011s sys 0.003s
I20260927 23:02:49.462312 24796 consensus_queue.cc:260] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 } } peers { permanent_uuid: "6e810acae7564d82a8b0436936759777" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43337 } } peers { permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 } }
I20260927 23:02:49.463706 24774 heartbeater.cc:503] Master 127.4.74.190:42019 was elected leader, sending a full tablet report...
I20260927 23:02:49.464176 24796 ts_tablet_manager.cc:1428] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260927 23:02:49.502841 24799 raft_consensus.cc:482] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:02:49.503209 24799 raft_consensus.cc:504] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 } } peers { permanent_uuid: "6e810acae7564d82a8b0436936759777" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43337 } } peers { permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 } }
I20260927 23:02:49.505151 24799 leader_election.cc:288] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe597dd90ac45fbbbac2ca97b77aafa (127.4.74.129:35431), af67e8ccc1ee410f8e817aea0514abb9 (127.4.74.131:41261)
I20260927 23:02:49.513077 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.505714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:43951 (local address 127.4.74.129:35431)
0927 23:02:49.506016 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.506026 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.507127 (+ 1101us) server_negotiation.cc:407] Connection header received
0927 23:02:49.507407 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.507421 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.507534 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.507665 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:49.508270 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.509076 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.511147 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.512018 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.512247 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.512336 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.512614 (+ 278us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.512760 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260927 23:02:49.513124 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.505732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:35431 (local address 127.4.74.130:43951)
0927 23:02:49.506981 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.506999 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:49.507228 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:49.507686 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:49.507692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:49.507719 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:49.508119 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.508128 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.509223 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.509228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.510920 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.510932 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.512247 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.512254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.512427 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.512452 (+ 25us) client_negotiation.cc:769] Sending connection context
0927 23:02:49.512581 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 23:02:49.512741 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1050,"thread_start_us":661,"threads_started":1}
I20260927 23:02:49.513537 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.505838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.74.130:60493 (local address 127.4.74.131:41261)
0927 23:02:49.506089 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.506098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.507476 (+ 1378us) server_negotiation.cc:407] Connection header received
0927 23:02:49.507801 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.507815 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.507893 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.508011 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 23:02:49.508688 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.509504 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.511603 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.512493 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.512723 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.512805 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.513098 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.513244 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260927 23:02:49.514137 24802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.505824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.131:41261 (local address 127.4.74.130:60493)
0927 23:02:49.507270 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.507289 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 23:02:49.507538 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:02:49.508034 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:02:49.508039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:02:49.508078 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 23:02:49.508491 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.508501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.509684 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.509696 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.511369 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:02:49.511383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.512724 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.512731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:02:49.512916 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.512945 (+ 29us) client_negotiation.cc:769] Sending connection context
0927 23:02:49.513072 (+ 127us) client_negotiation.cc:240] Negotiation successful
0927 23:02:49.513773 (+ 701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1313,"thread_start_us":781,"threads_started":1}
I20260927 23:02:49.514798 24468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2ed3020e98495291c2f4659fc21f40" candidate_uuid: "6e810acae7564d82a8b0436936759777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" is_pre_election: true
I20260927 23:02:49.515435 24468 raft_consensus.cc:2454] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e810acae7564d82a8b0436936759777 in term 0.
I20260927 23:02:49.515801 24729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2ed3020e98495291c2f4659fc21f40" candidate_uuid: "6e810acae7564d82a8b0436936759777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af67e8ccc1ee410f8e817aea0514abb9" is_pre_election: true
I20260927 23:02:49.516480 24534 leader_election.cc:302] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e810acae7564d82a8b0436936759777, efe597dd90ac45fbbbac2ca97b77aafa; no voters:
I20260927 23:02:49.516547 24729 raft_consensus.cc:2454] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e810acae7564d82a8b0436936759777 in term 0.
I20260927 23:02:49.516942 24799 raft_consensus.cc:2781] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:02:49.517110 24799 raft_consensus.cc:482] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:02:49.517237 24799 raft_consensus.cc:3037] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:49.519743 24799 raft_consensus.cc:504] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 } } peers { permanent_uuid: "6e810acae7564d82a8b0436936759777" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43337 } } peers { permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 } }
I20260927 23:02:49.520519 24799 leader_election.cc:288] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [CANDIDATE]: Term 1 election: Requested vote from peers efe597dd90ac45fbbbac2ca97b77aafa (127.4.74.129:35431), af67e8ccc1ee410f8e817aea0514abb9 (127.4.74.131:41261)
I20260927 23:02:49.521238 24729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2ed3020e98495291c2f4659fc21f40" candidate_uuid: "6e810acae7564d82a8b0436936759777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af67e8ccc1ee410f8e817aea0514abb9"
I20260927 23:02:49.521289 24468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2ed3020e98495291c2f4659fc21f40" candidate_uuid: "6e810acae7564d82a8b0436936759777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe597dd90ac45fbbbac2ca97b77aafa"
I20260927 23:02:49.521562 24729 raft_consensus.cc:3037] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:49.521551 24468 raft_consensus.cc:3037] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:49.525455 24729 raft_consensus.cc:2454] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e810acae7564d82a8b0436936759777 in term 1.
I20260927 23:02:49.525455 24468 raft_consensus.cc:2454] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e810acae7564d82a8b0436936759777 in term 1.
I20260927 23:02:49.526290 24532 leader_election.cc:302] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e810acae7564d82a8b0436936759777, af67e8ccc1ee410f8e817aea0514abb9; no voters:
I20260927 23:02:49.527061 24799 raft_consensus.cc:2781] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:49.527757 24799 raft_consensus.cc:686] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 1 LEADER]: Becoming Leader. State: Replica: 6e810acae7564d82a8b0436936759777, State: Running, Role: LEADER
I20260927 23:02:49.528317 24799 consensus_queue.cc:237] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 } } peers { permanent_uuid: "6e810acae7564d82a8b0436936759777" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43337 } } peers { permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 } }
I20260927 23:02:49.534428 24325 catalog_manager.cc:5495] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e810acae7564d82a8b0436936759777 (127.4.74.130). New cstate: current_term: 1 leader_uuid: "6e810acae7564d82a8b0436936759777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e810acae7564d82a8b0436936759777" member_type: VOTER last_known_addr { host: "127.4.74.130" port: 43337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 } health_report { overall_health: UNKNOWN } } }
W20260927 23:02:49.566133 24645 tablet.cc:2369] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:49.567484 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.560941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54608 (local address 127.4.74.130:43337)
0927 23:02:49.561233 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.561239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.561263 (+ 24us) server_negotiation.cc:407] Connection header received
0927 23:02:49.561466 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.561472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.561604 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.561770 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:02:49.562206 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.562984 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.564496 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.564956 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.565079 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.565534 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:02:49.566556 (+ 1022us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:02:49.566729 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.567117 (+ 388us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.567214 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260927 23:02:49.569748 24514 tablet.cc:2369] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 23:02:49.570802 24775 tablet.cc:2369] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:02:49.577831 24468 raft_consensus.cc:1261] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa [term 1 FOLLOWER]: Refusing update from remote peer 6e810acae7564d82a8b0436936759777: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:49.577831 24729 raft_consensus.cc:1261] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9 [term 1 FOLLOWER]: Refusing update from remote peer 6e810acae7564d82a8b0436936759777: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:02:49.579331 24803 consensus_queue.cc:1035] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af67e8ccc1ee410f8e817aea0514abb9" member_type: VOTER last_known_addr { host: "127.4.74.131" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:49.579828 24799 consensus_queue.cc:1035] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe597dd90ac45fbbbac2ca97b77aafa" member_type: VOTER last_known_addr { host: "127.4.74.129" port: 35431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:02:49.599273 24807 mvcc.cc:205] Tried to move back new op lower bound from 7334093494575927296 to 7334093494392258560. Current Snapshot: MvccSnapshot[applied={T|T < 7334093494575927296}]
I20260927 23:02:49.611150 24326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:49.611599 24326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 23:02:49.615851 24326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b8d41618d564ecf87e6c640609e373a: Sending DeleteTablet for 3 replicas of tablet 4a2ed3020e98495291c2f4659fc21f40
I20260927 23:02:49.617357 24448 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2ed3020e98495291c2f4659fc21f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:49 UTC) from {username='slave'} at 127.0.0.1:54618
I20260927 23:02:49.617321 24579 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2ed3020e98495291c2f4659fc21f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:49 UTC) from {username='slave'} at 127.0.0.1:54600
I20260927 23:02:49.618094 24709 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2ed3020e98495291c2f4659fc21f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:02:49 UTC) from {username='slave'} at 127.0.0.1:39140
I20260927 23:02:49.624276 24815 tablet_replica.cc:330] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: stopping tablet replica
I20260927 23:02:49.624910 24815 raft_consensus.cc:2229] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:49.625573 24815 raft_consensus.cc:2258] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:49.626093 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 24386
I20260927 23:02:49.628763 24816 tablet_replica.cc:330] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: stopping tablet replica
I20260927 23:02:49.629344 24816 raft_consensus.cc:2229] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:02:49.630281 24816 raft_consensus.cc:2258] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:49.633502 24815 ts_tablet_manager.cc:1905] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:49.638643 24817 tablet_replica.cc:330] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: stopping tablet replica
I20260927 23:02:49.639160 24817 raft_consensus.cc:2229] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:02:49.639777 24817 raft_consensus.cc:2258] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:02:49.642617 24817 ts_tablet_manager.cc:1905] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:49.643774 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.628057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54630 (local address 127.4.74.129:35431)
0927 23:02:49.628297 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.628307 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.632165 (+ 3858us) server_negotiation.cc:407] Connection header received
0927 23:02:49.632523 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.632532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.632691 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.632824 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:49.633825 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.635447 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.639699 (+ 4252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.640426 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.640560 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.642088 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:49.642096 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:49.642109 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:49.642748 (+ 639us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:49.642843 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:49.642870 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:49.643029 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.643131 (+ 102us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.643264 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 23:02:49.644582 24508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:49.648132 24816 ts_tablet_manager.cc:1905] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:02:49.651187 24817 ts_tablet_manager.cc:1918] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:49.651427 24817 log.cc:1196] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-2/wal/wals/4a2ed3020e98495291c2f4659fc21f40
I20260927 23:02:49.652132 24817 ts_tablet_manager.cc:1939] T 4a2ed3020e98495291c2f4659fc21f40 P af67e8ccc1ee410f8e817aea0514abb9: Deleting consensus metadata
I20260927 23:02:49.654948 24311 catalog_manager.cc:4841] TS af67e8ccc1ee410f8e817aea0514abb9 (127.4.74.131:41261): tablet 4a2ed3020e98495291c2f4659fc21f40 (table test-table [id=fdfcbc8e1efc46eebee9b5d73d50ba77]) successfully deleted
I20260927 23:02:49.660218 24816 ts_tablet_manager.cc:1918] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:49.660437 24816 log.cc:1196] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-1/wal/wals/4a2ed3020e98495291c2f4659fc21f40
I20260927 23:02:49.660962 24816 ts_tablet_manager.cc:1939] T 4a2ed3020e98495291c2f4659fc21f40 P 6e810acae7564d82a8b0436936759777: Deleting consensus metadata
I20260927 23:02:49.662600 24311 catalog_manager.cc:4841] TS 6e810acae7564d82a8b0436936759777 (127.4.74.130:43337): tablet 4a2ed3020e98495291c2f4659fc21f40 (table test-table [id=fdfcbc8e1efc46eebee9b5d73d50ba77]) successfully deleted
I20260927 23:02:49.765017 24815 ts_tablet_manager.cc:1918] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:02:49.765399 24815 log.cc:1196] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: Deleting WAL directory at /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790549821755302-4394-0/minicluster-data/ts-0/wal/wals/4a2ed3020e98495291c2f4659fc21f40
I20260927 23:02:49.765956 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 24386
I20260927 23:02:49.766114 24815 ts_tablet_manager.cc:1939] T 4a2ed3020e98495291c2f4659fc21f40 P efe597dd90ac45fbbbac2ca97b77aafa: Deleting consensus metadata
I20260927 23:02:49.785241 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 24517
W20260927 23:02:49.785203 24313 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.74.129:35431 (error 108)
I20260927 23:02:49.787776 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.787148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:35431 (local address 127.0.0.1:54644)
0927 23:02:49.787403 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.787572 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:35431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260927 23:02:49.788270 24313 catalog_manager.cc:4584] TS efe597dd90ac45fbbbac2ca97b77aafa (127.4.74.129:35431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a2ed3020e98495291c2f4659fc21f40: Network error: Client connection negotiation failed: client connection to 127.4.74.129:35431: connect: Connection refused (error 111)
I20260927 23:02:49.793783 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.786526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54610 (local address 127.4.74.130:43337)
0927 23:02:49.786799 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.786807 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.786837 (+ 30us) server_negotiation.cc:407] Connection header received
0927 23:02:49.787102 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.787107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.787235 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.787349 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:49.788077 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.789093 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.790394 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.790879 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.791021 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.792304 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:49.792312 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:49.792326 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:49.792967 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:49.793018 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:49.793039 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:49.793169 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.793288 (+ 119us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.793390 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260927 23:02:49.794766 24639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:49.842305 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.841573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:35431 (local address 127.0.0.1:54660)
0927 23:02:49.841909 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.842088 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:35431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260927 23:02:49.843688 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.843215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:35431 (local address 127.0.0.1:54662)
0927 23:02:49.843369 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.843453 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:35431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260927 23:02:49.905119 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.904332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:35431 (local address 127.0.0.1:54676)
0927 23:02:49.904610 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.904842 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:35431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260927 23:02:49.906344 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.905965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:35431 (local address 127.0.0.1:54692)
0927 23:02:49.906128 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.906213 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:35431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 23:02:49.916132 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 24517
I20260927 23:02:49.932667 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 24647
I20260927 23:02:49.942126 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.933693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39150 (local address 127.4.74.131:41261)
0927 23:02:49.933914 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:49.933918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:49.933941 (+ 23us) server_negotiation.cc:407] Connection header received
0927 23:02:49.934101 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:49.934106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:49.934225 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:49.934348 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:49.934920 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.936027 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.938032 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:49.938724 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:49.938905 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:49.940301 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:49.940312 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:49.940331 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:49.941147 (+ 816us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:49.941224 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:49.941254 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:49.941408 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:49.941585 (+ 177us) server_negotiation.cc:299] Negotiation successful
0927 23:02:49.941731 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260927 23:02:49.943042 24769 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:49.987655 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.986854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:35431 (local address 127.0.0.1:54700)
0927 23:02:49.987210 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.987329 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:35431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
I20260927 23:02:49.989152 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:49.988552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:35431 (local address 127.0.0.1:54716)
0927 23:02:49.988853 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:49.988995 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:35431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260927 23:02:50.042992 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 24647
I20260927 23:02:50.060307 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 24296
I20260927 23:02:50.061655 24357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:50.185714 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:50.184742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.129:35431 (local address 127.0.0.1:54722)
0927 23:02:50.185268 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0927 23:02:50.185444 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.129:35431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247}
I20260927 23:02:50.186154 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 24296
2026-09-27T23:02:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3266 ms)
[----------] 6 tests from EncryptionPolicyTest (24650 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:02:50 dist-test-slave-5pd3 krb5kdc[24826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:02:50 dist-test-slave-5pd3 krb5kdc[24826](info): set up 2 sockets
Sep 27 23:02:50 dist-test-slave-5pd3 krb5kdc[24826](info): commencing operation
krb5kdc: starting...
W20260927 23:02:51.380412 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.008s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:02:51 dist-test-slave-5pd3 krb5kdc[24826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:02:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:02:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:02:51.693194 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:52.066733 24839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:52.067057 24839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:52.067195 24839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:52.074980 24839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:52.075119 24839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:52.075167 24839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:52.075276 24839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:52.075361 24839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:52.086097 24839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46863
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:52.087529 24839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:52.088924 24839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:52.102923 24844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:52.102941 24845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:52.103866 24847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:52.105888 24839 server_base.cc:1034] running on GCE node
I20260927 23:02:52.107191 24839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:52.109263 24839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:52.110562 24839 hybrid_clock.cc:648] HybridClock initialized: now 1790550172110498 us; error 55 us; skew 500 ppm
I20260927 23:02:52.111111 24839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:52 dist-test-slave-5pd3 krb5kdc[24826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550172, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:52.115394 24839 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:02:52.117509 24839 webserver.cc:460] Webserver started at http://127.0.0.1:37513/ using document root <none> and password file <none>
I20260927 23:02:52.118345 24839 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:52.118474 24839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:52.118804 24839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:52.122514 24839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "7f239332f855462e95830f6423abba70"
format_stamp: "Formatted at 2026-09-27 23:02:52 on dist-test-slave-5pd3"
server_key: "8b20b3c81c22cceb730d0cbe60d38038"
server_key_iv: "95e160e772ac315773dde31ee3a4fb3a"
server_key_version: "encryptionkey@0"
I20260927 23:02:52.123389 24839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "7f239332f855462e95830f6423abba70"
format_stamp: "Formatted at 2026-09-27 23:02:52 on dist-test-slave-5pd3"
server_key: "8b20b3c81c22cceb730d0cbe60d38038"
server_key_iv: "95e160e772ac315773dde31ee3a4fb3a"
server_key_version: "encryptionkey@0"
I20260927 23:02:52.130162 24839 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:02:52.135133 24854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:52.137141 24839 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260927 23:02:52.137408 24839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "7f239332f855462e95830f6423abba70"
format_stamp: "Formatted at 2026-09-27 23:02:52 on dist-test-slave-5pd3"
server_key: "8b20b3c81c22cceb730d0cbe60d38038"
server_key_iv: "95e160e772ac315773dde31ee3a4fb3a"
server_key_version: "encryptionkey@0"
I20260927 23:02:52.137663 24839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:52.155234 24839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:52.160763 24839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:52.161140 24839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:52.185707 24839 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46863
I20260927 23:02:52.185770 24905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46863 every 8 connection(s)
I20260927 23:02:52.187403 24839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:52.193492 24906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:52.196744 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 24839
I20260927 23:02:52.196992 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:02:52.197721 4394 external_mini_cluster.cc:1428] Setting key a10a99e23608e6c1592726944af9aa12
I20260927 23:02:52.213263 24906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70: Bootstrap starting.
I20260927 23:02:52.218145 24906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70: Neither blocks nor log segments found. Creating new log.
Sep 27 23:02:52 dist-test-slave-5pd3 krb5kdc[24826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 23:02:52.219772 24906 log.cc:824] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:52.225128 24906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70: No bootstrap required, opened a new log
I20260927 23:02:52.232016 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:52.201120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42201 (local address 127.0.0.1:46863)
0927 23:02:52.202414 (+ 1294us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:52.202443 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:52.202541 (+ 98us) server_negotiation.cc:407] Connection header received
0927 23:02:52.205777 (+ 3236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:52.205828 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:52.206639 (+ 811us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:52.207322 (+ 683us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:52.209126 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:52.211611 (+ 2485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:52.213187 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:52.214438 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:52.214706 (+ 268us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:52.220099 (+ 5393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:52.220114 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:52.220143 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:52.220213 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:52.225285 (+ 5072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:52.226529 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:52.226541 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:52.226553 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:52.226844 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:52.227453 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:52.227459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:52.227463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:52.228504 (+ 1041us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:52.228928 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:52.229500 (+ 572us) server_negotiation.cc:299] Negotiation successful
0927 23:02:52.230020 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":919,"thread_start_us":574,"threads_started":1}
I20260927 23:02:52.237386 24906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f239332f855462e95830f6423abba70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46863 } }
I20260927 23:02:52.237922 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:02:52.238018 24906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:52.238133 24906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f239332f855462e95830f6423abba70, State: Initialized, Role: FOLLOWER
I20260927 23:02:52.238915 24906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f239332f855462e95830f6423abba70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46863 } }
I20260927 23:02:52.239203 24906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:52.239353 24906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:52.239609 24906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:52.242755 24906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f239332f855462e95830f6423abba70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46863 } }
I20260927 23:02:52.243605 24906 leader_election.cc:302] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f239332f855462e95830f6423abba70; no voters:
I20260927 23:02:52.244654 24906 leader_election.cc:288] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:02:52.244906 24915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:52.247488 24906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:52.249233 24915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [term 1 LEADER]: Becoming Leader. State: Replica: 7f239332f855462e95830f6423abba70, State: Running, Role: LEADER
I20260927 23:02:52.250260 24915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f239332f855462e95830f6423abba70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46863 } }
I20260927 23:02:52.255332 24918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f239332f855462e95830f6423abba70. Latest consensus state: current_term: 1 leader_uuid: "7f239332f855462e95830f6423abba70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f239332f855462e95830f6423abba70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46863 } } }
I20260927 23:02:52.255414 24917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f239332f855462e95830f6423abba70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f239332f855462e95830f6423abba70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46863 } } }
I20260927 23:02:52.255973 24918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:52.255971 24917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:52.257900 24923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:52.267755 24923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:52.272478 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:52.244412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38239 (local address 127.0.0.1:46863)
0927 23:02:52.244671 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:52.244684 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:52.253250 (+ 8566us) server_negotiation.cc:407] Connection header received
0927 23:02:52.253492 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:52.253497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:52.253611 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:52.253719 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:52.255826 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:52.257066 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:52.258526 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:52.259435 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:52.260344 (+ 909us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:52.263118 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:52.263125 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:52.263130 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:52.263180 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:52.267478 (+ 4298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:52.268827 (+ 1349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:52.268833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:52.268837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:52.269006 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:52.269665 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:52.269671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:52.269675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:52.270117 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:52.271646 (+ 1529us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:52.271844 (+ 198us) server_negotiation.cc:299] Negotiation successful
0927 23:02:52.272065 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260927 23:02:52.286768 24923 catalog_manager.cc:1347] Generated new cluster ID: 2da77e6c500c4622a9647a572457074e
I20260927 23:02:52.286950 24923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:52.299777 24923 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:02:52.301469 24923 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:52.311196 24923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f239332f855462e95830f6423abba70: Generated new TSK 0
I20260927 23:02:52.312633 24923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:02:52.331475 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 24839
I20260927 23:02:52.333190 24901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:52.434821 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 24839
W20260927 23:02:52.456858 24824 connection.cc:535] client connection to 127.0.0.1:46863 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46863): Connection reset by peer (error 104)
2026-09-27T23:02:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2241 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:02:52 dist-test-slave-5pd3 krb5kdc[24938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:02:52 dist-test-slave-5pd3 krb5kdc[24938](info): set up 2 sockets
Sep 27 23:02:52 dist-test-slave-5pd3 krb5kdc[24938](info): commencing operation
krb5kdc: starting...
W20260927 23:02:53.628472 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:02:53 dist-test-slave-5pd3 krb5kdc[24938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:02:53.944530 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:54.347000 24951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:54.347404 24951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:54.347659 24951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:54.355293 24951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:54.355481 24951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:54.355628 24951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:54.355751 24951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:54.355849 24951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:54.366324 24951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:54.368079 24951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:54.369503 24951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:54.383675 24957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:54.383764 24956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:54.383764 24959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:54.385941 24951 server_base.cc:1034] running on GCE node
I20260927 23:02:54.386896 24951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:54.388715 24951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:54.389981 24951 hybrid_clock.cc:648] HybridClock initialized: now 1790550174389921 us; error 59 us; skew 500 ppm
I20260927 23:02:54.390431 24951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:54 dist-test-slave-5pd3 krb5kdc[24938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:54.401954 24951 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:02:54.404233 24951 webserver.cc:460] Webserver started at http://127.0.0.1:44683/ using document root <none> and password file <none>
I20260927 23:02:54.405078 24951 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:54.405247 24951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:54.405586 24951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:54.408948 24951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "4b65ca1995574a4ca1f940017c2e347e"
format_stamp: "Formatted at 2026-09-27 23:02:54 on dist-test-slave-5pd3"
server_key: "a4b3f311e8a6115979f6e2d591f41bbb"
server_key_iv: "6fc0b5b0ab042d1a2cd11896abc77731"
server_key_version: "encryptionkey@0"
I20260927 23:02:54.409847 24951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "4b65ca1995574a4ca1f940017c2e347e"
format_stamp: "Formatted at 2026-09-27 23:02:54 on dist-test-slave-5pd3"
server_key: "a4b3f311e8a6115979f6e2d591f41bbb"
server_key_iv: "6fc0b5b0ab042d1a2cd11896abc77731"
server_key_version: "encryptionkey@0"
I20260927 23:02:54.416201 24951 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260927 23:02:54.420740 24966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:54.422614 24951 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:02:54.422937 24951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "4b65ca1995574a4ca1f940017c2e347e"
format_stamp: "Formatted at 2026-09-27 23:02:54 on dist-test-slave-5pd3"
server_key: "a4b3f311e8a6115979f6e2d591f41bbb"
server_key_iv: "6fc0b5b0ab042d1a2cd11896abc77731"
server_key_version: "encryptionkey@0"
I20260927 23:02:54.423216 24951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:54.446300 24951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:54.451293 24951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:54.451695 24951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:54.474364 24951 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42459
I20260927 23:02:54.474474 25017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42459 every 8 connection(s)
I20260927 23:02:54.476091 24951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:54.479095 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 24951
I20260927 23:02:54.479378 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:02:54.480085 4394 external_mini_cluster.cc:1428] Setting key 8e99d93bc28c3b7353dcc8ffbbde3191
I20260927 23:02:54.482055 25018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:54.495711 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:54.482487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57355 (local address 127.0.0.1:42459)
0927 23:02:54.483448 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:54.483465 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:54.483521 (+ 56us) server_negotiation.cc:407] Connection header received
0927 23:02:54.485455 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:54.485522 (+ 67us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:54.486069 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:54.486599 (+ 530us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:54.487273 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:54.488836 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:54.490282 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:54.491097 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:54.491317 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:54.492827 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:54.492836 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:54.492854 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:54.493566 (+ 712us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:54.493666 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:54.493702 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:54.493917 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:54.494232 (+ 315us) server_negotiation.cc:299] Negotiation successful
0927 23:02:54.494447 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":404,"threads_started":1}
I20260927 23:02:54.497476 25018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e: Bootstrap starting.
I20260927 23:02:54.500660 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:02:54.502297 25018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:54.503911 25018 log.cc:824] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:54.508086 25018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e: No bootstrap required, opened a new log
I20260927 23:02:54.513424 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:54.505737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53731 (local address 127.0.0.1:42459)
0927 23:02:54.505957 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:54.505966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:54.506534 (+ 568us) server_negotiation.cc:407] Connection header received
0927 23:02:54.506871 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:54.506876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:54.506957 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:54.507080 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:54.507821 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:54.508777 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:54.510271 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:54.510806 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:54.510994 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:54.512460 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:54.512468 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:54.512472 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:54.512497 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:54.512549 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:54.512571 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:54.512717 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:54.512922 (+ 205us) server_negotiation.cc:299] Negotiation successful
0927 23:02:54.512996 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 23:02:54.518126 25018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b65ca1995574a4ca1f940017c2e347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260927 23:02:54.518613 25018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:54.518785 25018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b65ca1995574a4ca1f940017c2e347e, State: Initialized, Role: FOLLOWER
I20260927 23:02:54.519683 25018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b65ca1995574a4ca1f940017c2e347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260927 23:02:54.519981 25018 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:54.520161 25018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:54.520416 25018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:54.523082 25018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b65ca1995574a4ca1f940017c2e347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260927 23:02:54.523778 25018 leader_election.cc:302] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b65ca1995574a4ca1f940017c2e347e; no voters:
I20260927 23:02:54.524528 25018 leader_election.cc:288] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:02:54.524801 25028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:54.527005 25018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:54.532138 25028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [term 1 LEADER]: Becoming Leader. State: Replica: 4b65ca1995574a4ca1f940017c2e347e, State: Running, Role: LEADER
I20260927 23:02:54.533269 25028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b65ca1995574a4ca1f940017c2e347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260927 23:02:54.537079 25029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b65ca1995574a4ca1f940017c2e347e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b65ca1995574a4ca1f940017c2e347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } } }
I20260927 23:02:54.537578 25029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:54.537616 25030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b65ca1995574a4ca1f940017c2e347e. Latest consensus state: current_term: 1 leader_uuid: "4b65ca1995574a4ca1f940017c2e347e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b65ca1995574a4ca1f940017c2e347e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } } }
I20260927 23:02:54.538233 25030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e [sys.catalog]: This master's current role is: LEADER
W20260927 23:02:54.545651 25043 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:02:54.545854 25043 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:02:54.546070 25042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:54.548900 25042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:54.562320 25042 catalog_manager.cc:1347] Generated new cluster ID: 374254a1b7d14b7ab10d6c84dd6bbc11
I20260927 23:02:54.562551 25042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:54.603386 25042 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:02:54.605547 25042 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:54.615419 25042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b65ca1995574a4ca1f940017c2e347e: Generated new TSK 0
I20260927 23:02:54.616835 25042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:02:54.638139 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 24951
W20260927 23:02:54.640470 25013 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57355
W20260927 23:02:54.641522 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 24951: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:02:54.641809 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 24951
W20260927 23:02:54.663251 24936 connection.cc:535] client connection to 127.0.0.1:42459 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42459): Connection reset by peer (error 104)
2026-09-27T23:02:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2205 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:02:54 dist-test-slave-5pd3 krb5kdc[25049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:02:54 dist-test-slave-5pd3 krb5kdc[25049](info): set up 2 sockets
Sep 27 23:02:54 dist-test-slave-5pd3 krb5kdc[25049](info): commencing operation
krb5kdc: starting...
W20260927 23:02:55.804584 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:02:56 dist-test-slave-5pd3 krb5kdc[25049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:02:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:02:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:02:56.128190 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44679
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:56.495654 25062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:56.496001 25062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:56.496101 25062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:56.504689 25062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:56.504864 25062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:56.504917 25062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:56.504983 25062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:56.505023 25062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:56.516530 25062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44679
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:56.517983 25062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:56.519301 25062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:56.534418 25067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:56.534421 25068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:56.535429 25062 server_base.cc:1034] running on GCE node
W20260927 23:02:56.535499 25070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:56.536801 25062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:56.538980 25062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:56.540266 25062 hybrid_clock.cc:648] HybridClock initialized: now 1790550176540213 us; error 55 us; skew 500 ppm
I20260927 23:02:56.540764 25062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:56 dist-test-slave-5pd3 krb5kdc[25049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:56.544749 25062 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:02:56.546743 25062 webserver.cc:460] Webserver started at http://127.0.0.1:43151/ using document root <none> and password file <none>
I20260927 23:02:56.547751 25062 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:56.547891 25062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:56.548215 25062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:56.552526 25062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "e41822b3c9484643a6d6787563a10c56"
format_stamp: "Formatted at 2026-09-27 23:02:56 on dist-test-slave-5pd3"
server_key: "9fdc592677ed44eeaf0e8f02ddc1ac4d"
server_key_iv: "da464833860c00b4ae1aff8a40a53257"
server_key_version: "encryptionkey@0"
I20260927 23:02:56.553537 25062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "e41822b3c9484643a6d6787563a10c56"
format_stamp: "Formatted at 2026-09-27 23:02:56 on dist-test-slave-5pd3"
server_key: "9fdc592677ed44eeaf0e8f02ddc1ac4d"
server_key_iv: "da464833860c00b4ae1aff8a40a53257"
server_key_version: "encryptionkey@0"
I20260927 23:02:56.562036 25062 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260927 23:02:56.567310 25077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:56.569854 25062 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260927 23:02:56.570201 25062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "e41822b3c9484643a6d6787563a10c56"
format_stamp: "Formatted at 2026-09-27 23:02:56 on dist-test-slave-5pd3"
server_key: "9fdc592677ed44eeaf0e8f02ddc1ac4d"
server_key_iv: "da464833860c00b4ae1aff8a40a53257"
server_key_version: "encryptionkey@0"
I20260927 23:02:56.570482 25062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:56.578477 25062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:56.584828 25062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:56.585259 25062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:56.609606 25062 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44679
I20260927 23:02:56.609658 25128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44679 every 8 connection(s)
I20260927 23:02:56.611799 25062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:56.616551 25129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:02:56.620574 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 25062
I20260927 23:02:56.620908 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:02:56.621467 4394 external_mini_cluster.cc:1428] Setting key b5f6730c5dc76ec48524a528f7eb8667
I20260927 23:02:56.632100 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:56.623957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41445 (local address 127.0.0.1:44679)
0927 23:02:56.625137 (+ 1180us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:56.625156 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:56.625226 (+ 70us) server_negotiation.cc:407] Connection header received
0927 23:02:56.627341 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:56.627382 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:56.628014 (+ 632us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:56.628543 (+ 529us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:56.629876 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:56.629885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:56.629904 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:56.629926 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:56.630022 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:56.630064 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:56.630329 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:56.630465 (+ 136us) server_negotiation.cc:299] Negotiation successful
0927 23:02:56.630682 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":863,"thread_start_us":527,"threads_started":1}
I20260927 23:02:56.632550 25129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56: Bootstrap starting.
I20260927 23:02:56.636478 25129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:56.637820 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:02:56.637852 25129 log.cc:824] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:56.641943 25129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56: No bootstrap required, opened a new log
I20260927 23:02:56.651170 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:56.644272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54745 (local address 127.0.0.1:44679)
0927 23:02:56.644501 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:56.644506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:56.648449 (+ 3943us) server_negotiation.cc:407] Connection header received
0927 23:02:56.648725 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:56.648730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:56.648809 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:56.648947 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:56.650321 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:56.650327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:56.650330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:02:56.650352 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:56.650407 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:02:56.650426 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:56.650592 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:56.650712 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 23:02:56.650803 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 23:02:56.653237 25129 raft_consensus.cc:348] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41822b3c9484643a6d6787563a10c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44679 } }
I20260927 23:02:56.653959 25129 raft_consensus.cc:374] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:56.654081 25129 raft_consensus.cc:729] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e41822b3c9484643a6d6787563a10c56, State: Initialized, Role: FOLLOWER
I20260927 23:02:56.654749 25129 consensus_queue.cc:260] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41822b3c9484643a6d6787563a10c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44679 } }
I20260927 23:02:56.655045 25129 raft_consensus.cc:388] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:56.655212 25129 raft_consensus.cc:482] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:56.655406 25129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:56.658034 25129 raft_consensus.cc:504] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41822b3c9484643a6d6787563a10c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44679 } }
I20260927 23:02:56.658841 25129 leader_election.cc:302] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e41822b3c9484643a6d6787563a10c56; no voters:
I20260927 23:02:56.659735 25129 leader_election.cc:288] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:02:56.659938 25139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:56.660864 25139 raft_consensus.cc:686] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [term 1 LEADER]: Becoming Leader. State: Replica: e41822b3c9484643a6d6787563a10c56, State: Running, Role: LEADER
I20260927 23:02:56.661655 25139 consensus_queue.cc:237] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41822b3c9484643a6d6787563a10c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44679 } }
I20260927 23:02:56.662182 25129 sys_catalog.cc:564] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:56.665690 25140 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e41822b3c9484643a6d6787563a10c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41822b3c9484643a6d6787563a10c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44679 } } }
I20260927 23:02:56.666359 25140 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:56.667421 25141 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e41822b3c9484643a6d6787563a10c56. Latest consensus state: current_term: 1 leader_uuid: "e41822b3c9484643a6d6787563a10c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41822b3c9484643a6d6787563a10c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44679 } } }
I20260927 23:02:56.668000 25141 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:56.672945 25146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:56.680526 25146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:56.693208 25146 catalog_manager.cc:1347] Generated new cluster ID: a31de4c7585e45fdb4b9332cf03099cd
I20260927 23:02:56.693378 25146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:56.716953 25146 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:02:56.717526 25146 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:56.733242 25146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e41822b3c9484643a6d6787563a10c56: Generated new TSK 0
I20260927 23:02:56.734947 25146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:02:56.749128 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25062
W20260927 23:02:56.750890 25124 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41445
W20260927 23:02:56.751796 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25062: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:02:56.751997 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 25062
2026-09-27T23:02:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2107 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:02:56 dist-test-slave-5pd3 krb5kdc[25160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:02:56 dist-test-slave-5pd3 krb5kdc[25160](info): set up 2 sockets
Sep 27 23:02:56 dist-test-slave-5pd3 krb5kdc[25160](info): commencing operation
krb5kdc: starting...
W20260927 23:02:57.936638 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:02:58 dist-test-slave-5pd3 krb5kdc[25160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:02:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:02:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:02:58.265482 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:02:58.630538 25173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:02:58.630901 25173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:02:58.631067 25173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:02:58.639330 25173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:02:58.639765 25173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:02:58.639886 25173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:02:58.639987 25173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:02:58.640062 25173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:02:58.651985 25173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:02:58.653491 25173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:02:58.654939 25173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:02:58.669433 25178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:58.669489 25179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:02:58.669351 25181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:02:58.670817 25173 server_base.cc:1034] running on GCE node
I20260927 23:02:58.671900 25173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:02:58.673990 25173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:02:58.677464 25173 hybrid_clock.cc:648] HybridClock initialized: now 1790550178677382 us; error 70 us; skew 500 ppm
I20260927 23:02:58.677947 25173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:02:58 dist-test-slave-5pd3 krb5kdc[25160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550178, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:02:58.681859 25173 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:02:58.683939 25173 webserver.cc:460] Webserver started at http://127.0.0.1:32935/ using document root <none> and password file <none>
I20260927 23:02:58.684824 25173 fs_manager.cc:359] Metadata directory not provided
I20260927 23:02:58.684953 25173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:02:58.685282 25173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:02:58.688714 25173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "529c6afad5054761b5e1fba3ea998e93"
format_stamp: "Formatted at 2026-09-27 23:02:58 on dist-test-slave-5pd3"
server_key: "6fec38cfb06c805dbcf874e8dc424576"
server_key_iv: "a2600fe8048a3c3a627e61489a77a396"
server_key_version: "encryptionkey@0"
I20260927 23:02:58.689687 25173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "529c6afad5054761b5e1fba3ea998e93"
format_stamp: "Formatted at 2026-09-27 23:02:58 on dist-test-slave-5pd3"
server_key: "6fec38cfb06c805dbcf874e8dc424576"
server_key_iv: "a2600fe8048a3c3a627e61489a77a396"
server_key_version: "encryptionkey@0"
I20260927 23:02:58.696497 25173 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260927 23:02:58.701316 25188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:02:58.703460 25173 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260927 23:02:58.703833 25173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "529c6afad5054761b5e1fba3ea998e93"
format_stamp: "Formatted at 2026-09-27 23:02:58 on dist-test-slave-5pd3"
server_key: "6fec38cfb06c805dbcf874e8dc424576"
server_key_iv: "a2600fe8048a3c3a627e61489a77a396"
server_key_version: "encryptionkey@0"
I20260927 23:02:58.704103 25173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:02:58.717779 25173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:02:58.723796 25173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:02:58.724145 25173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:02:58.747584 25173 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45121
I20260927 23:02:58.747714 25239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45121 every 8 connection(s)
I20260927 23:02:58.749203 25173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:02:58.749922 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 25173
I20260927 23:02:58.750188 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:02:58.750931 4394 external_mini_cluster.cc:1428] Setting key 45c612e59a46aa7796d25ec2f6686f5c
I20260927 23:02:58.754628 25240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 23:02:58 dist-test-slave-5pd3 krb5kdc[25160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 23:02:58.772104 25240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93: Bootstrap starting.
I20260927 23:02:58.775858 25240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93: Neither blocks nor log segments found. Creating new log.
I20260927 23:02:58.777307 25240 log.cc:824] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93: Log is configured to *not* fsync() on all Append() calls
I20260927 23:02:58.781834 25240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93: No bootstrap required, opened a new log
I20260927 23:02:58.785562 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:58.753396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54021 (local address 127.0.0.1:45121)
0927 23:02:58.754793 (+ 1397us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:58.754815 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:58.754885 (+ 70us) server_negotiation.cc:407] Connection header received
0927 23:02:58.757058 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:58.757091 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:58.757731 (+ 640us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:58.758253 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:58.760558 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:58.762628 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:58.765241 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:58.766340 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:58.766604 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:58.773962 (+ 7358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:58.773975 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:58.773998 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:58.774078 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:58.779129 (+ 5051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:58.780523 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:58.780532 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:58.780547 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:58.780830 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:58.781366 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:58.781370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:58.781372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:58.782187 (+ 815us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:58.782467 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:58.783309 (+ 842us) server_negotiation.cc:299] Negotiation successful
0927 23:02:58.784030 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1041,"thread_start_us":739,"threads_started":1}
I20260927 23:02:58.791388 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:02:58.792265 25240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529c6afad5054761b5e1fba3ea998e93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260927 23:02:58.792721 25240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:02:58.792830 25240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529c6afad5054761b5e1fba3ea998e93, State: Initialized, Role: FOLLOWER
I20260927 23:02:58.793496 25240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529c6afad5054761b5e1fba3ea998e93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260927 23:02:58.793747 25240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:02:58.793901 25240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:02:58.794098 25240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:02:58.796626 25240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529c6afad5054761b5e1fba3ea998e93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260927 23:02:58.797243 25240 leader_election.cc:302] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 529c6afad5054761b5e1fba3ea998e93; no voters:
I20260927 23:02:58.798266 25240 leader_election.cc:288] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:02:58.798465 25249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:02:58.800442 25249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [term 1 LEADER]: Becoming Leader. State: Replica: 529c6afad5054761b5e1fba3ea998e93, State: Running, Role: LEADER
I20260927 23:02:58.801318 25249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529c6afad5054761b5e1fba3ea998e93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260927 23:02:58.801605 25240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:02:58.804978 25252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 529c6afad5054761b5e1fba3ea998e93. Latest consensus state: current_term: 1 leader_uuid: "529c6afad5054761b5e1fba3ea998e93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529c6afad5054761b5e1fba3ea998e93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } } }
I20260927 23:02:58.805545 25252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:58.806181 25251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "529c6afad5054761b5e1fba3ea998e93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529c6afad5054761b5e1fba3ea998e93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } } }
I20260927 23:02:58.806525 25251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93 [sys.catalog]: This master's current role is: LEADER
I20260927 23:02:58.819312 25256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:02:58.827440 25256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:02:58.830778 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:02:58.797896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36383 (local address 127.0.0.1:45121)
0927 23:02:58.798189 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0927 23:02:58.798196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:02:58.804567 (+ 6371us) server_negotiation.cc:407] Connection header received
0927 23:02:58.804821 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:02:58.804828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:02:58.804947 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:02:58.805090 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:02:58.808032 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:58.809611 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:58.812412 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:02:58.813462 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:02:58.813604 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:02:58.818289 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:02:58.818300 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:02:58.818308 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:02:58.818378 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:02:58.824402 (+ 6024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:58.825837 (+ 1435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:58.825846 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:58.825868 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:58.826214 (+ 346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:02:58.826815 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:02:58.826824 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:02:58.826830 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:02:58.827466 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:02:58.827963 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0927 23:02:58.829789 (+ 1826us) server_negotiation.cc:299] Negotiation successful
0927 23:02:58.830181 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260927 23:02:58.850021 25256 catalog_manager.cc:1347] Generated new cluster ID: f2419da616f1478d800c07e0bff504cb
I20260927 23:02:58.850183 25256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:02:58.865113 25256 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:02:58.866588 25256 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:02:58.873449 25256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 529c6afad5054761b5e1fba3ea998e93: Generated new TSK 0
I20260927 23:02:58.875116 25256 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:02:58.890817 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25173
I20260927 23:02:58.893333 25235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:02:58.994006 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 25173
2026-09-27T23:02:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2246 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:02:59 dist-test-slave-5pd3 krb5kdc[25272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:02:59 dist-test-slave-5pd3 krb5kdc[25272](info): set up 2 sockets
Sep 27 23:02:59 dist-test-slave-5pd3 krb5kdc[25272](info): commencing operation
krb5kdc: starting...
W20260927 23:03:00.160101 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:00 dist-test-slave-5pd3 krb5kdc[25272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:03:00.491842 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:00.945839 25285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:00.946271 25285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:00.946457 25285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:00.955299 25285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:00.955484 25285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:00.955621 25285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:00.955732 25285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:00.955813 25285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:00.967724 25285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:00.969558 25285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:00.971290 25285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:00.986274 25293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:00.986410 25290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:00.987144 25285 server_base.cc:1034] running on GCE node
W20260927 23:03:00.986580 25291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:00.988564 25285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:00.990863 25285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:00.992144 25285 hybrid_clock.cc:648] HybridClock initialized: now 1790550180992074 us; error 65 us; skew 500 ppm
I20260927 23:03:00.992672 25285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:00 dist-test-slave-5pd3 krb5kdc[25272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550180, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:01.002794 25285 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:03:01.005191 25285 webserver.cc:460] Webserver started at http://127.0.0.1:39117/ using document root <none> and password file <none>
I20260927 23:03:01.006249 25285 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:01.006397 25285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:01.006855 25285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:01.012497 25285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "ccada30ec51b466c9988044a6e547014"
format_stamp: "Formatted at 2026-09-27 23:03:01 on dist-test-slave-5pd3"
server_key: "32f1f01eafd6c5e22eebffb3f46d9eaa"
server_key_iv: "37629d24736b59d01132dc84f66d10bc"
server_key_version: "encryptionkey@0"
I20260927 23:03:01.013801 25285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "ccada30ec51b466c9988044a6e547014"
format_stamp: "Formatted at 2026-09-27 23:03:01 on dist-test-slave-5pd3"
server_key: "32f1f01eafd6c5e22eebffb3f46d9eaa"
server_key_iv: "37629d24736b59d01132dc84f66d10bc"
server_key_version: "encryptionkey@0"
I20260927 23:03:01.021955 25285 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260927 23:03:01.027966 25300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:01.029915 25285 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:03:01.030153 25285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "ccada30ec51b466c9988044a6e547014"
format_stamp: "Formatted at 2026-09-27 23:03:01 on dist-test-slave-5pd3"
server_key: "32f1f01eafd6c5e22eebffb3f46d9eaa"
server_key_iv: "37629d24736b59d01132dc84f66d10bc"
server_key_version: "encryptionkey@0"
I20260927 23:03:01.030406 25285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:01.044188 25285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:01.049854 25285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:01.050287 25285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:01.074193 25285 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36313
I20260927 23:03:01.074265 25351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36313 every 8 connection(s)
I20260927 23:03:01.076098 25285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:01.079130 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 25285
I20260927 23:03:01.079384 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:01.080052 4394 external_mini_cluster.cc:1428] Setting key 18dbda3485fcefc804c1d599de47b480
I20260927 23:03:01.082212 25352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:01.100812 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:01.082850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46059 (local address 127.0.0.1:36313)
0927 23:03:01.084126 (+ 1276us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:01.084145 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:01.084724 (+ 579us) server_negotiation.cc:407] Connection header received
0927 23:03:01.086811 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:01.086851 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:01.087698 (+ 847us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:01.088328 (+ 630us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:01.089032 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:01.091318 (+ 2286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:01.093658 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:01.094863 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:01.095110 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:01.096895 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:01.096912 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:01.096940 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:01.097962 (+ 1022us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:01.098105 (+ 143us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:01.098150 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:01.098311 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:01.098641 (+ 330us) server_negotiation.cc:299] Negotiation successful
0927 23:03:01.099067 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":923,"thread_start_us":543,"threads_started":1}
I20260927 23:03:01.101486 25352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014: Bootstrap starting.
I20260927 23:03:01.106702 25352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:01.106850 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:01.108451 25352 log.cc:824] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:01.113718 25352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014: No bootstrap required, opened a new log
I20260927 23:03:01.121577 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:01.113171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55993 (local address 127.0.0.1:36313)
0927 23:03:01.113379 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:01.113383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:01.113787 (+ 404us) server_negotiation.cc:407] Connection header received
0927 23:03:01.114134 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:01.114139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:01.114222 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:01.114377 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:01.115100 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:01.116163 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:01.118403 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:01.118940 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:01.119088 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:01.120605 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:01.120613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:01.120617 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:01.120645 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:01.120716 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:01.120742 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:01.120849 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:01.121029 (+ 180us) server_negotiation.cc:299] Negotiation successful
0927 23:03:01.121132 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260927 23:03:01.125310 25352 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccada30ec51b466c9988044a6e547014" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36313 } }
I20260927 23:03:01.126065 25352 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:01.126246 25352 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccada30ec51b466c9988044a6e547014, State: Initialized, Role: FOLLOWER
I20260927 23:03:01.127272 25352 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccada30ec51b466c9988044a6e547014" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36313 } }
I20260927 23:03:01.127686 25352 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:01.127868 25352 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:01.128093 25352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:01.131422 25352 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccada30ec51b466c9988044a6e547014" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36313 } }
I20260927 23:03:01.132264 25352 leader_election.cc:302] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccada30ec51b466c9988044a6e547014; no voters:
I20260927 23:03:01.133097 25352 leader_election.cc:288] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:01.133405 25362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:01.134749 25362 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [term 1 LEADER]: Becoming Leader. State: Replica: ccada30ec51b466c9988044a6e547014, State: Running, Role: LEADER
I20260927 23:03:01.136037 25362 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccada30ec51b466c9988044a6e547014" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36313 } }
I20260927 23:03:01.136503 25352 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:01.139423 25363 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccada30ec51b466c9988044a6e547014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccada30ec51b466c9988044a6e547014" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36313 } } }
I20260927 23:03:01.141098 25363 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:01.140117 25364 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccada30ec51b466c9988044a6e547014. Latest consensus state: current_term: 1 leader_uuid: "ccada30ec51b466c9988044a6e547014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccada30ec51b466c9988044a6e547014" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36313 } } }
I20260927 23:03:01.141630 25364 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:01.142956 25371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:01.151698 25371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:01.167286 25371 catalog_manager.cc:1347] Generated new cluster ID: 62f9128d89134285b83b1692a7034916
I20260927 23:03:01.167461 25371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:01.198762 25371 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:01.201354 25371 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:01.215971 25371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccada30ec51b466c9988044a6e547014: Generated new TSK 0
I20260927 23:03:01.217078 25371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:01.235188 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25285
W20260927 23:03:01.236629 25347 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46059
W20260927 23:03:01.237723 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25285: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:03:01.238193 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 25285
2026-09-27T23:03:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2240 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:01 dist-test-slave-5pd3 krb5kdc[25383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:01 dist-test-slave-5pd3 krb5kdc[25383](info): set up 2 sockets
Sep 27 23:03:01 dist-test-slave-5pd3 krb5kdc[25383](info): commencing operation
krb5kdc: starting...
W20260927 23:03:02.408702 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:02 dist-test-slave-5pd3 krb5kdc[25383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:03:02.728451 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:03.111406 25396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:03.111886 25396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:03.112041 25396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:03.120441 25396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:03.120574 25396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:03.120653 25396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:03.120784 25396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:03.120853 25396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:03.131325 25396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:03.132848 25396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:03.134264 25396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:03.148655 25401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:03.148799 25402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:03.148705 25404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:03.150625 25396 server_base.cc:1034] running on GCE node
I20260927 23:03:03.151701 25396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:03.153355 25396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:03.154640 25396 hybrid_clock.cc:648] HybridClock initialized: now 1790550183154586 us; error 57 us; skew 500 ppm
I20260927 23:03:03.155093 25396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:03 dist-test-slave-5pd3 krb5kdc[25383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550183, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:03.159037 25396 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:03:03.161063 25396 webserver.cc:460] Webserver started at http://127.0.0.1:33383/ using document root <none> and password file <none>
I20260927 23:03:03.161886 25396 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:03.162055 25396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:03.162390 25396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:03.165709 25396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "e8e2825fecd549b1aaf1ceffe39750ea"
format_stamp: "Formatted at 2026-09-27 23:03:03 on dist-test-slave-5pd3"
server_key: "ce227ae708c8931477619b2a1f29d3ec"
server_key_iv: "202c7ce7c8c243058731f28689c4ee78"
server_key_version: "encryptionkey@0"
I20260927 23:03:03.166558 25396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "e8e2825fecd549b1aaf1ceffe39750ea"
format_stamp: "Formatted at 2026-09-27 23:03:03 on dist-test-slave-5pd3"
server_key: "ce227ae708c8931477619b2a1f29d3ec"
server_key_iv: "202c7ce7c8c243058731f28689c4ee78"
server_key_version: "encryptionkey@0"
I20260927 23:03:03.172519 25396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260927 23:03:03.176761 25411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:03.178758 25396 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:03:03.179035 25396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "e8e2825fecd549b1aaf1ceffe39750ea"
format_stamp: "Formatted at 2026-09-27 23:03:03 on dist-test-slave-5pd3"
server_key: "ce227ae708c8931477619b2a1f29d3ec"
server_key_iv: "202c7ce7c8c243058731f28689c4ee78"
server_key_version: "encryptionkey@0"
I20260927 23:03:03.179299 25396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:03.186681 25396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:03.191431 25396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:03.191793 25396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:03.215620 25396 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42029
I20260927 23:03:03.215741 25462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42029 every 8 connection(s)
I20260927 23:03:03.217347 25396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:03.222620 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 25396
I20260927 23:03:03.222874 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:03.223495 4394 external_mini_cluster.cc:1428] Setting key e40850cd22e2b93e5d4bb1003503f9c6
I20260927 23:03:03.223595 25463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:03.234356 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:03.226021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40781 (local address 127.0.0.1:42029)
0927 23:03:03.227067 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:03.227082 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:03.227135 (+ 53us) server_negotiation.cc:407] Connection header received
0927 23:03:03.230083 (+ 2948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:03.230116 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:03.230600 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:03.231131 (+ 531us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:03.232480 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:03.232487 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:03.232507 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:03.232524 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:03.232590 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:03.232611 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:03.232904 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:03.233075 (+ 171us) server_negotiation.cc:299] Negotiation successful
0927 23:03:03.233291 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":431,"threads_started":1}
I20260927 23:03:03.239221 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:03.241475 25463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea: Bootstrap starting.
I20260927 23:03:03.247368 25463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:03.249360 25463 log.cc:824] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:03.249733 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:03.245914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53245 (local address 127.0.0.1:42029)
0927 23:03:03.246366 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:03.246374 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:03.247125 (+ 751us) server_negotiation.cc:407] Connection header received
0927 23:03:03.247326 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:03.247335 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:03.247438 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:03.247670 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:03.248831 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:03.248840 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:03.248846 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:03.248878 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:03.248948 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:03.248974 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:03.249160 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:03.249268 (+ 108us) server_negotiation.cc:299] Negotiation successful
0927 23:03:03.249357 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260927 23:03:03.254012 25463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea: No bootstrap required, opened a new log
I20260927 23:03:03.264621 25463 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e2825fecd549b1aaf1ceffe39750ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260927 23:03:03.265247 25463 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:03.265369 25463 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e2825fecd549b1aaf1ceffe39750ea, State: Initialized, Role: FOLLOWER
I20260927 23:03:03.266109 25463 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e2825fecd549b1aaf1ceffe39750ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260927 23:03:03.266362 25463 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:03.266547 25463 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:03.266778 25463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:03.269575 25463 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e2825fecd549b1aaf1ceffe39750ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260927 23:03:03.270273 25463 leader_election.cc:302] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8e2825fecd549b1aaf1ceffe39750ea; no voters:
I20260927 23:03:03.271066 25463 leader_election.cc:288] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:03.271322 25473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:03.274183 25473 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [term 1 LEADER]: Becoming Leader. State: Replica: e8e2825fecd549b1aaf1ceffe39750ea, State: Running, Role: LEADER
I20260927 23:03:03.274927 25473 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e2825fecd549b1aaf1ceffe39750ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260927 23:03:03.275626 25463 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:03.278363 25474 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8e2825fecd549b1aaf1ceffe39750ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e2825fecd549b1aaf1ceffe39750ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } } }
I20260927 23:03:03.278801 25474 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:03.279305 25475 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8e2825fecd549b1aaf1ceffe39750ea. Latest consensus state: current_term: 1 leader_uuid: "e8e2825fecd549b1aaf1ceffe39750ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e2825fecd549b1aaf1ceffe39750ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } } }
I20260927 23:03:03.279706 25475 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:03.283424 25480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:03.289575 25480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:03.302480 25480 catalog_manager.cc:1347] Generated new cluster ID: 540635d881044573b9f27e759656930e
I20260927 23:03:03.302703 25480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:03.337525 25480 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:03.338172 25480 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:03.350149 25480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8e2825fecd549b1aaf1ceffe39750ea: Generated new TSK 0
I20260927 23:03:03.351444 25480 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:03.374531 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25396
W20260927 23:03:03.376080 25458 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40781
W20260927 23:03:03.377040 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25396: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:03:03.377215 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 25396
2026-09-27T23:03:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2137 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:03 dist-test-slave-5pd3 krb5kdc[25494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:03 dist-test-slave-5pd3 krb5kdc[25494](info): set up 2 sockets
Sep 27 23:03:03 dist-test-slave-5pd3 krb5kdc[25494](info): commencing operation
krb5kdc: starting...
W20260927 23:03:04.536322 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:04 dist-test-slave-5pd3 krb5kdc[25494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:03:04.853734 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41035
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:05.288822 25507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:05.289245 25507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:05.289356 25507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:05.298497 25507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:05.298673 25507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:05.298727 25507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:05.298789 25507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:05.298838 25507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:05.310967 25507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:05.312729 25507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:05.314739 25507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:05.330662 25513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:05.330745 25512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:05.330745 25515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:05.332202 25507 server_base.cc:1034] running on GCE node
I20260927 23:03:05.333271 25507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:05.335244 25507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:05.336503 25507 hybrid_clock.cc:648] HybridClock initialized: now 1790550185336451 us; error 85 us; skew 500 ppm
I20260927 23:03:05.337030 25507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:05 dist-test-slave-5pd3 krb5kdc[25494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550185, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:05.341321 25507 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:03:05.343693 25507 webserver.cc:460] Webserver started at http://127.0.0.1:35465/ using document root <none> and password file <none>
I20260927 23:03:05.344671 25507 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:05.344842 25507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:05.345232 25507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:05.349618 25507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "a7dd116651ee404cacd3c2c849f1eea1"
format_stamp: "Formatted at 2026-09-27 23:03:05 on dist-test-slave-5pd3"
server_key: "7bd9b28f417925edbf0ee17e1fe1264d"
server_key_iv: "b683ef3a725e32bfa2582fed84547a70"
server_key_version: "encryptionkey@0"
I20260927 23:03:05.350725 25507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "a7dd116651ee404cacd3c2c849f1eea1"
format_stamp: "Formatted at 2026-09-27 23:03:05 on dist-test-slave-5pd3"
server_key: "7bd9b28f417925edbf0ee17e1fe1264d"
server_key_iv: "b683ef3a725e32bfa2582fed84547a70"
server_key_version: "encryptionkey@0"
I20260927 23:03:05.357463 25507 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260927 23:03:05.361732 25522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:05.363759 25507 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:03:05.364060 25507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "a7dd116651ee404cacd3c2c849f1eea1"
format_stamp: "Formatted at 2026-09-27 23:03:05 on dist-test-slave-5pd3"
server_key: "7bd9b28f417925edbf0ee17e1fe1264d"
server_key_iv: "b683ef3a725e32bfa2582fed84547a70"
server_key_version: "encryptionkey@0"
I20260927 23:03:05.364310 25507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:05.390568 25507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:05.395412 25507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:05.395758 25507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:05.418960 25507 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41035
I20260927 23:03:05.419020 25573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41035 every 8 connection(s)
I20260927 23:03:05.420765 25507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:05.421118 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 25507
I20260927 23:03:05.421322 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:05.422053 4394 external_mini_cluster.cc:1428] Setting key 51f398a56b530fc79524cb5435cb0c67
I20260927 23:03:05.427147 25574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 23:03:05 dist-test-slave-5pd3 krb5kdc[25494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 23:03:05.442730 25574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1: Bootstrap starting.
I20260927 23:03:05.448136 25574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:05.449908 25574 log.cc:824] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:05.452687 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:05.424475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:41525 (local address 127.0.0.1:41035)
0927 23:03:05.425449 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:05.425470 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:05.425537 (+ 67us) server_negotiation.cc:407] Connection header received
0927 23:03:05.428223 (+ 2686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:05.428259 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:05.428915 (+ 656us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:05.429465 (+ 550us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:05.431085 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:05.433372 (+ 2287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:05.434920 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:05.436171 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:05.436424 (+ 253us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:05.441564 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:05.441578 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:05.441605 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:05.441697 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:05.446464 (+ 4767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:05.447772 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:05.447786 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:05.447800 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:05.448089 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:05.448737 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:05.448745 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:05.448748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:05.449690 (+ 942us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:05.450070 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:05.450706 (+ 636us) server_negotiation.cc:299] Negotiation successful
0927 23:03:05.451134 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":401,"threads_started":1}
I20260927 23:03:05.454358 25574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1: No bootstrap required, opened a new log
I20260927 23:03:05.458042 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:05.464440 25574 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7dd116651ee404cacd3c2c849f1eea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41035 } }
I20260927 23:03:05.464982 25574 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:05.465108 25574 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7dd116651ee404cacd3c2c849f1eea1, State: Initialized, Role: FOLLOWER
I20260927 23:03:05.465937 25574 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7dd116651ee404cacd3c2c849f1eea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41035 } }
I20260927 23:03:05.466296 25574 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:05.466468 25574 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:05.466686 25574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:05.470114 25574 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7dd116651ee404cacd3c2c849f1eea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41035 } }
I20260927 23:03:05.470821 25574 leader_election.cc:302] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7dd116651ee404cacd3c2c849f1eea1; no voters:
I20260927 23:03:05.471668 25574 leader_election.cc:288] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:05.471818 25584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:05.475134 25574 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:05.475888 25584 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [term 1 LEADER]: Becoming Leader. State: Replica: a7dd116651ee404cacd3c2c849f1eea1, State: Running, Role: LEADER
I20260927 23:03:05.476817 25584 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7dd116651ee404cacd3c2c849f1eea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41035 } }
I20260927 23:03:05.480396 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:05.463203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:40279 (local address 127.0.0.1:41035)
0927 23:03:05.463451 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:05.463459 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:05.463807 (+ 348us) server_negotiation.cc:407] Connection header received
0927 23:03:05.463966 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:05.463972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:05.464078 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:05.464233 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:05.465906 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:05.467177 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:05.468689 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:05.469482 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:05.469621 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:05.472941 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:05.472950 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:05.472957 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:05.473006 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:05.476603 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:05.477816 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:05.477823 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:05.477832 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:05.478019 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:05.478544 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:05.478548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:05.478550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:05.478916 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:05.479198 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:05.479839 (+ 641us) server_negotiation.cc:299] Negotiation successful
0927 23:03:05.480036 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260927 23:03:05.486166 25585 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7dd116651ee404cacd3c2c849f1eea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7dd116651ee404cacd3c2c849f1eea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41035 } } }
I20260927 23:03:05.486977 25585 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:05.486092 25586 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7dd116651ee404cacd3c2c849f1eea1. Latest consensus state: current_term: 1 leader_uuid: "a7dd116651ee404cacd3c2c849f1eea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7dd116651ee404cacd3c2c849f1eea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41035 } } }
I20260927 23:03:05.487677 25586 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:05.493607 25594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:05.501783 25594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:05.514578 25594 catalog_manager.cc:1347] Generated new cluster ID: ef34cb18affe41998d3efcb3ffe9daaf
I20260927 23:03:05.514799 25594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:05.549118 25594 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:05.550951 25594 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:05.559461 25594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7dd116651ee404cacd3c2c849f1eea1: Generated new TSK 0
I20260927 23:03:05.560775 25594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:05.574144 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25507
I20260927 23:03:05.576339 25569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:03:05.686363 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 25507
W20260927 23:03:05.708120 25492 connection.cc:535] client connection to 127.0.0.1:41035 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41035): Connection reset by peer (error 104)
2026-09-27T23:03:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:05 dist-test-slave-5pd3 krb5kdc[25606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:05 dist-test-slave-5pd3 krb5kdc[25606](info): set up 2 sockets
Sep 27 23:03:05 dist-test-slave-5pd3 krb5kdc[25606](info): commencing operation
krb5kdc: starting...
W20260927 23:03:06.820744 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:07 dist-test-slave-5pd3 krb5kdc[25606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:03:07.144390 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33571
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:07.563086 25619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:07.563426 25619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:07.563612 25619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:07.571467 25619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:07.571707 25619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:07.571801 25619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:07.571900 25619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:07.571990 25619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:07.582554 25619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:07.584131 25619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:07.585615 25619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:07.600412 25624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:07.600241 25627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:07.600236 25625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:07.601325 25619 server_base.cc:1034] running on GCE node
I20260927 23:03:07.602396 25619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:07.604260 25619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:07.605571 25619 hybrid_clock.cc:648] HybridClock initialized: now 1790550187605524 us; error 54 us; skew 500 ppm
I20260927 23:03:07.606076 25619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:07 dist-test-slave-5pd3 krb5kdc[25606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550187, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:07.616901 25619 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:03:07.618988 25619 webserver.cc:460] Webserver started at http://127.0.0.1:33071/ using document root <none> and password file <none>
I20260927 23:03:07.620007 25619 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:07.620184 25619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:07.620534 25619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:07.624069 25619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "a662ee5f0f0745358655af52f3ecb5e6"
format_stamp: "Formatted at 2026-09-27 23:03:07 on dist-test-slave-5pd3"
server_key: "78f9d557b121b06b6e78f97bbf9a953f"
server_key_iv: "6eca076db44e0001ef2d9bc1adaaf5d1"
server_key_version: "encryptionkey@0"
I20260927 23:03:07.624991 25619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "a662ee5f0f0745358655af52f3ecb5e6"
format_stamp: "Formatted at 2026-09-27 23:03:07 on dist-test-slave-5pd3"
server_key: "78f9d557b121b06b6e78f97bbf9a953f"
server_key_iv: "6eca076db44e0001ef2d9bc1adaaf5d1"
server_key_version: "encryptionkey@0"
I20260927 23:03:07.631722 25619 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 23:03:07.636297 25634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:07.638237 25619 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:03:07.638525 25619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "a662ee5f0f0745358655af52f3ecb5e6"
format_stamp: "Formatted at 2026-09-27 23:03:07 on dist-test-slave-5pd3"
server_key: "78f9d557b121b06b6e78f97bbf9a953f"
server_key_iv: "6eca076db44e0001ef2d9bc1adaaf5d1"
server_key_version: "encryptionkey@0"
I20260927 23:03:07.638773 25619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:07.662523 25619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:07.667850 25619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:07.668181 25619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:07.694614 25619 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33571
I20260927 23:03:07.694686 25685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33571 every 8 connection(s)
I20260927 23:03:07.696390 25619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:07.698470 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 25619
I20260927 23:03:07.698693 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:07.699137 4394 external_mini_cluster.cc:1428] Setting key 52d3ff7d9b0b9a414452d35195b0bf15
I20260927 23:03:07.702813 25686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:07.722492 25689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:07.701819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:41993 (local address 127.0.0.1:33571)
0927 23:03:07.703185 (+ 1366us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:07.703216 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:07.703540 (+ 324us) server_negotiation.cc:407] Connection header received
0927 23:03:07.706502 (+ 2962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:07.706544 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:07.707353 (+ 809us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:07.708494 (+ 1141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:07.709442 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:07.712305 (+ 2863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:07.714342 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:07.715712 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:07.715978 (+ 266us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:07.718080 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:07.718099 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:07.718139 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:07.719259 (+ 1120us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:07.719421 (+ 162us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:07.719481 (+ 60us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:07.719811 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:07.720341 (+ 530us) server_negotiation.cc:299] Negotiation successful
0927 23:03:07.720650 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":980,"thread_start_us":565,"threads_started":1}
I20260927 23:03:07.723948 25686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6: Bootstrap starting.
I20260927 23:03:07.728552 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:07.729563 25686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:07.731791 25686 log.cc:824] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:07.736075 25686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6: No bootstrap required, opened a new log
I20260927 23:03:07.744517 25689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:07.735686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:34129 (local address 127.0.0.1:33571)
0927 23:03:07.735999 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:07.736007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:07.736546 (+ 539us) server_negotiation.cc:407] Connection header received
0927 23:03:07.736816 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:07.736821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:07.736909 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:07.737050 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:07.737731 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:07.738675 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:07.740812 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:07.741375 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:07.741504 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:07.743316 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:07.743322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:07.743326 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:07.743369 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:07.743439 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:07.743461 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:07.743704 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:07.744004 (+ 300us) server_negotiation.cc:299] Negotiation successful
0927 23:03:07.744096 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260927 23:03:07.747219 25686 raft_consensus.cc:348] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a662ee5f0f0745358655af52f3ecb5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33571 } }
I20260927 23:03:07.747769 25686 raft_consensus.cc:374] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:07.747905 25686 raft_consensus.cc:729] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a662ee5f0f0745358655af52f3ecb5e6, State: Initialized, Role: FOLLOWER
I20260927 23:03:07.748603 25686 consensus_queue.cc:260] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a662ee5f0f0745358655af52f3ecb5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33571 } }
I20260927 23:03:07.748832 25686 raft_consensus.cc:388] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:07.749015 25686 raft_consensus.cc:482] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:07.749222 25686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:07.752370 25686 raft_consensus.cc:504] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a662ee5f0f0745358655af52f3ecb5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33571 } }
I20260927 23:03:07.753059 25686 leader_election.cc:302] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a662ee5f0f0745358655af52f3ecb5e6; no voters:
I20260927 23:03:07.754101 25686 leader_election.cc:288] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:07.754314 25696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:07.755676 25696 raft_consensus.cc:686] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [term 1 LEADER]: Becoming Leader. State: Replica: a662ee5f0f0745358655af52f3ecb5e6, State: Running, Role: LEADER
I20260927 23:03:07.756476 25696 consensus_queue.cc:237] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a662ee5f0f0745358655af52f3ecb5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33571 } }
I20260927 23:03:07.756992 25686 sys_catalog.cc:564] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:07.761217 25697 sys_catalog.cc:455] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a662ee5f0f0745358655af52f3ecb5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a662ee5f0f0745358655af52f3ecb5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33571 } } }
I20260927 23:03:07.761758 25697 sys_catalog.cc:458] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:07.762481 25698 sys_catalog.cc:455] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a662ee5f0f0745358655af52f3ecb5e6. Latest consensus state: current_term: 1 leader_uuid: "a662ee5f0f0745358655af52f3ecb5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a662ee5f0f0745358655af52f3ecb5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33571 } } }
I20260927 23:03:07.762825 25698 sys_catalog.cc:458] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:07.772058 25704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:07.778119 25704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:07.790248 25704 catalog_manager.cc:1347] Generated new cluster ID: d2ca455894e64205b3a8bc6fc977b21d
I20260927 23:03:07.790531 25704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:07.812911 25704 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:07.815021 25704 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:07.827961 25704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a662ee5f0f0745358655af52f3ecb5e6: Generated new TSK 0
I20260927 23:03:07.829366 25704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:07.840523 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25619
W20260927 23:03:07.842123 25681 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.246:41993
W20260927 23:03:07.843235 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25619: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:03:07.843417 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 25619
W20260927 23:03:07.864461 25604 connection.cc:535] client connection to 127.0.0.1:33571 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33571): Connection reset by peer (error 104)
2026-09-27T23:03:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2156 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:07 dist-test-slave-5pd3 krb5kdc[25718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:07 dist-test-slave-5pd3 krb5kdc[25718](info): set up 2 sockets
Sep 27 23:03:07 dist-test-slave-5pd3 krb5kdc[25718](info): commencing operation
krb5kdc: starting...
W20260927 23:03:09.012496 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.002s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:09 dist-test-slave-5pd3 krb5kdc[25718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:03:09.328277 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:09.694391 25731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:09.694715 25731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:09.694875 25731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:09.702795 25731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:09.702950 25731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:09.703001 25731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:09.703130 25731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:09.703209 25731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:09.713770 25731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:09.715229 25731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:09.716662 25731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:09.729979 25736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:09.730165 25737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:09.730710 25739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:09.731587 25731 server_base.cc:1034] running on GCE node
I20260927 23:03:09.732487 25731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:09.734098 25731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:09.735348 25731 hybrid_clock.cc:648] HybridClock initialized: now 1790550189735284 us; error 62 us; skew 500 ppm
I20260927 23:03:09.735791 25731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:09 dist-test-slave-5pd3 krb5kdc[25718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550189, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:09.739686 25731 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:03:09.741660 25731 webserver.cc:460] Webserver started at http://127.0.0.1:37145/ using document root <none> and password file <none>
I20260927 23:03:09.742451 25731 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:09.742616 25731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:09.742960 25731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:09.746805 25731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "02f9e43b8fe44a2595dbb43adeed25e8"
format_stamp: "Formatted at 2026-09-27 23:03:09 on dist-test-slave-5pd3"
server_key: "e325025dd157dc6ab4f94cf060a49932"
server_key_iv: "19e7cdfe4bacb46f9021f8d3b48179b0"
server_key_version: "encryptionkey@0"
I20260927 23:03:09.747833 25731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "02f9e43b8fe44a2595dbb43adeed25e8"
format_stamp: "Formatted at 2026-09-27 23:03:09 on dist-test-slave-5pd3"
server_key: "e325025dd157dc6ab4f94cf060a49932"
server_key_iv: "19e7cdfe4bacb46f9021f8d3b48179b0"
server_key_version: "encryptionkey@0"
I20260927 23:03:09.754269 25731 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:03:09.758988 25746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:09.760886 25731 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260927 23:03:09.761128 25731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "02f9e43b8fe44a2595dbb43adeed25e8"
format_stamp: "Formatted at 2026-09-27 23:03:09 on dist-test-slave-5pd3"
server_key: "e325025dd157dc6ab4f94cf060a49932"
server_key_iv: "19e7cdfe4bacb46f9021f8d3b48179b0"
server_key_version: "encryptionkey@0"
I20260927 23:03:09.761389 25731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:09.768229 25731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:09.773394 25731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:09.773768 25731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:09.796526 25731 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45647
I20260927 23:03:09.796648 25797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45647 every 8 connection(s)
I20260927 23:03:09.798187 25731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:09.801800 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 25731
I20260927 23:03:09.802031 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:09.802663 4394 external_mini_cluster.cc:1428] Setting key c90f2877fb7df6409ed366da4a8eb318
I20260927 23:03:09.803807 25798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:09.812076 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:09.805055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:46113 (local address 127.0.0.1:45647)
0927 23:03:09.805983 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:09.805996 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:09.806042 (+ 46us) server_negotiation.cc:407] Connection header received
0927 23:03:09.807925 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:09.807952 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:09.808393 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:09.808782 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:09.810206 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:09.810213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:09.810225 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:09.810262 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:09.810363 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:09.810389 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:09.810599 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:09.810704 (+ 105us) server_negotiation.cc:299] Negotiation successful
0927 23:03:09.810945 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":417,"threads_started":1}
I20260927 23:03:09.817034 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:09.821501 25798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8: Bootstrap starting.
I20260927 23:03:09.826763 25798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:09.827306 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:09.824030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:53947 (local address 127.0.0.1:45647)
0927 23:03:09.824331 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:09.824339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:09.824904 (+ 565us) server_negotiation.cc:407] Connection header received
0927 23:03:09.825073 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:09.825080 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:09.825162 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:09.825327 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:09.826373 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:09.826379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:09.826388 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:09.826411 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:09.826476 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:09.826503 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:09.826676 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:09.826871 (+ 195us) server_negotiation.cc:299] Negotiation successful
0927 23:03:09.826964 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260927 23:03:09.828589 25798 log.cc:824] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:09.832582 25798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8: No bootstrap required, opened a new log
I20260927 23:03:09.842727 25798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f9e43b8fe44a2595dbb43adeed25e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45647 } }
I20260927 23:03:09.843263 25798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:09.843339 25798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02f9e43b8fe44a2595dbb43adeed25e8, State: Initialized, Role: FOLLOWER
I20260927 23:03:09.844146 25798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f9e43b8fe44a2595dbb43adeed25e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45647 } }
I20260927 23:03:09.844410 25798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:09.844527 25798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:09.844679 25798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:09.847275 25798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f9e43b8fe44a2595dbb43adeed25e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45647 } }
I20260927 23:03:09.848026 25798 leader_election.cc:302] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02f9e43b8fe44a2595dbb43adeed25e8; no voters:
I20260927 23:03:09.848937 25798 leader_election.cc:288] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:09.849237 25808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:09.850608 25808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [term 1 LEADER]: Becoming Leader. State: Replica: 02f9e43b8fe44a2595dbb43adeed25e8, State: Running, Role: LEADER
I20260927 23:03:09.851326 25808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f9e43b8fe44a2595dbb43adeed25e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45647 } }
I20260927 23:03:09.851786 25798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:09.854559 25810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02f9e43b8fe44a2595dbb43adeed25e8. Latest consensus state: current_term: 1 leader_uuid: "02f9e43b8fe44a2595dbb43adeed25e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f9e43b8fe44a2595dbb43adeed25e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45647 } } }
I20260927 23:03:09.855135 25810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:09.854745 25809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02f9e43b8fe44a2595dbb43adeed25e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f9e43b8fe44a2595dbb43adeed25e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45647 } } }
I20260927 23:03:09.855513 25809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:09.856479 25816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:09.864377 25816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:09.876098 25816 catalog_manager.cc:1347] Generated new cluster ID: ef9536bd6507411fa0b6f4c47c8a8736
I20260927 23:03:09.876246 25816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:09.896687 25816 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:09.897500 25816 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:09.913534 25816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02f9e43b8fe44a2595dbb43adeed25e8: Generated new TSK 0
I20260927 23:03:09.915215 25816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:09.922670 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25731
W20260927 23:03:09.924062 25793 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.246:46113
W20260927 23:03:09.924952 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25731: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:03:09.925096 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 25731
2026-09-27T23:03:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2082 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:10 dist-test-slave-5pd3 krb5kdc[25829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:10 dist-test-slave-5pd3 krb5kdc[25829](info): set up 2 sockets
Sep 27 23:03:10 dist-test-slave-5pd3 krb5kdc[25829](info): commencing operation
krb5kdc: starting...
W20260927 23:03:11.112591 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:11 dist-test-slave-5pd3 krb5kdc[25829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:03:11.437278 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:11.789583 25842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:11.790207 25842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:11.790473 25842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:11.798177 25842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:11.798329 25842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:11.798422 25842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:11.798534 25842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:11.798648 25842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:11.809290 25842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43745
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:11.810762 25842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:11.812167 25842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:11.827131 25847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:11.827030 25850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:11.827085 25848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:11.828239 25842 server_base.cc:1034] running on GCE node
I20260927 23:03:11.829208 25842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:11.831931 25842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:11.835413 25842 hybrid_clock.cc:648] HybridClock initialized: now 1790550191835272 us; error 106 us; skew 500 ppm
I20260927 23:03:11.836063 25842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:11 dist-test-slave-5pd3 krb5kdc[25829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550191, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:11.841105 25842 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:03:11.842962 25842 webserver.cc:460] Webserver started at http://127.0.0.1:33669/ using document root <none> and password file <none>
I20260927 23:03:11.843824 25842 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:11.843946 25842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:11.844209 25842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:11.847349 25842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "0851daa181a74554a417eb0d6b77c9a8"
format_stamp: "Formatted at 2026-09-27 23:03:11 on dist-test-slave-5pd3"
server_key: "3638aee09b79ad63a766a5820c479b84"
server_key_iv: "e98950ffd29737221ed832bd2b2ead92"
server_key_version: "encryptionkey@0"
I20260927 23:03:11.848218 25842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "0851daa181a74554a417eb0d6b77c9a8"
format_stamp: "Formatted at 2026-09-27 23:03:11 on dist-test-slave-5pd3"
server_key: "3638aee09b79ad63a766a5820c479b84"
server_key_iv: "e98950ffd29737221ed832bd2b2ead92"
server_key_version: "encryptionkey@0"
I20260927 23:03:11.854493 25842 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:03:11.859124 25857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:11.861148 25842 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:03:11.861366 25842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "0851daa181a74554a417eb0d6b77c9a8"
format_stamp: "Formatted at 2026-09-27 23:03:11 on dist-test-slave-5pd3"
server_key: "3638aee09b79ad63a766a5820c479b84"
server_key_iv: "e98950ffd29737221ed832bd2b2ead92"
server_key_version: "encryptionkey@0"
I20260927 23:03:11.861575 25842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:11.919782 25842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:11.924566 25842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:11.924824 25842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:11.947717 25842 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43745
I20260927 23:03:11.947737 25908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43745 every 8 connection(s)
I20260927 23:03:11.949275 25842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:11.951484 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 25842
I20260927 23:03:11.951747 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:11.952445 4394 external_mini_cluster.cc:1428] Setting key 1c1284cab15387498d4c8fa8266db1ae
I20260927 23:03:11.955873 25909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 23:03:11 dist-test-slave-5pd3 krb5kdc[25829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 23:03:11.972458 25909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8: Bootstrap starting.
I20260927 23:03:11.977895 25909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:11.979521 25909 log.cc:824] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:11.981670 25912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:11.955132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:42025 (local address 127.0.0.1:43745)
0927 23:03:11.956242 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:11.956278 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:11.956329 (+ 51us) server_negotiation.cc:407] Connection header received
0927 23:03:11.958199 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:11.958227 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:11.958713 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:11.959177 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:11.960895 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:11.963475 (+ 2580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:11.965138 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:11.966471 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:11.966752 (+ 281us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:11.972144 (+ 5392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:11.972153 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:11.972173 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:11.972240 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:11.975982 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:11.977449 (+ 1467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:11.977457 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:11.977465 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:11.977683 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:11.978291 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:11.978295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:11.978297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:11.978989 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:11.979277 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:11.980030 (+ 753us) server_negotiation.cc:299] Negotiation successful
0927 23:03:11.980468 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":454,"threads_started":1}
I20260927 23:03:11.983482 25909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8: No bootstrap required, opened a new log
I20260927 23:03:11.986140 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:11.996058 25909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0851daa181a74554a417eb0d6b77c9a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } }
I20260927 23:03:11.996834 25909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:11.996966 25909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0851daa181a74554a417eb0d6b77c9a8, State: Initialized, Role: FOLLOWER
I20260927 23:03:11.997941 25909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0851daa181a74554a417eb0d6b77c9a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } }
I20260927 23:03:11.998270 25909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:11.998467 25909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:11.998694 25909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:12.002497 25909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0851daa181a74554a417eb0d6b77c9a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } }
I20260927 23:03:12.003398 25909 leader_election.cc:302] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0851daa181a74554a417eb0d6b77c9a8; no voters:
I20260927 23:03:12.004395 25909 leader_election.cc:288] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:12.004609 25919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:12.007874 25919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [term 1 LEADER]: Becoming Leader. State: Replica: 0851daa181a74554a417eb0d6b77c9a8, State: Running, Role: LEADER
I20260927 23:03:12.008133 25909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:12.008685 25919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0851daa181a74554a417eb0d6b77c9a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } }
I20260927 23:03:12.012864 25921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0851daa181a74554a417eb0d6b77c9a8. Latest consensus state: current_term: 1 leader_uuid: "0851daa181a74554a417eb0d6b77c9a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0851daa181a74554a417eb0d6b77c9a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } } }
I20260927 23:03:12.013423 25921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:12.014276 25920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0851daa181a74554a417eb0d6b77c9a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0851daa181a74554a417eb0d6b77c9a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43745 } } }
I20260927 23:03:12.014794 25920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:12.021436 25912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:11.992793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:37553 (local address 127.0.0.1:43745)
0927 23:03:11.993144 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:11.993175 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:11.994157 (+ 982us) server_negotiation.cc:407] Connection header received
0927 23:03:11.994638 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:11.994643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:11.994746 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:11.994870 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:11.997534 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:11.998576 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:12.000752 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:12.001302 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:12.001443 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:12.005416 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:12.005427 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:12.005436 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:12.005504 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:12.011118 (+ 5614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:12.013396 (+ 2278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:12.013415 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:12.013422 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:12.013883 (+ 461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:12.015684 (+ 1801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:12.015695 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:12.015702 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:12.016406 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:12.016954 (+ 548us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:12.020336 (+ 3382us) server_negotiation.cc:299] Negotiation successful
0927 23:03:12.020738 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260927 23:03:12.025251 25926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:12.031168 25926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:12.042702 25926 catalog_manager.cc:1347] Generated new cluster ID: f3ae8df1f17e446f85068d6a1f5aee65
I20260927 23:03:12.042868 25926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:12.053395 25926 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:12.055076 25926 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:12.066185 25926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0851daa181a74554a417eb0d6b77c9a8: Generated new TSK 0
I20260927 23:03:12.067318 25926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:12.075973 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25842
I20260927 23:03:12.077762 25904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:03:12.177872 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 25842
W20260927 23:03:12.196621 25827 connection.cc:535] client connection to 127.0.0.1:43745 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43745): Connection reset by peer (error 104)
2026-09-27T23:03:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2249 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:12 dist-test-slave-5pd3 krb5kdc[25942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:12 dist-test-slave-5pd3 krb5kdc[25942](info): set up 2 sockets
Sep 27 23:03:12 dist-test-slave-5pd3 krb5kdc[25942](info): commencing operation
krb5kdc: starting...
W20260927 23:03:13.384299 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.007s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:13 dist-test-slave-5pd3 krb5kdc[25942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:03:13.707927 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35711
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:14.072400 25955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:14.072716 25955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:14.072872 25955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:14.080267 25955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:14.080392 25955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:14.080483 25955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:14.080593 25955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:14.080693 25955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:14.091053 25955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35711
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:14.092434 25955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:14.093822 25955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:14.111696 25960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:14.111754 25961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:14.111985 25963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:14.112654 25955 server_base.cc:1034] running on GCE node
I20260927 23:03:14.113715 25955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:14.115854 25955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:14.117141 25955 hybrid_clock.cc:648] HybridClock initialized: now 1790550194117055 us; error 82 us; skew 500 ppm
I20260927 23:03:14.117630 25955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:14 dist-test-slave-5pd3 krb5kdc[25942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550194, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:14.129328 25955 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:03:14.131743 25955 webserver.cc:460] Webserver started at http://127.0.0.1:37631/ using document root <none> and password file <none>
I20260927 23:03:14.132712 25955 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:14.132921 25955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:14.133332 25955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:14.137794 25955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "8fe0073c69c04389ba0c9cf50a24c89e"
format_stamp: "Formatted at 2026-09-27 23:03:14 on dist-test-slave-5pd3"
server_key: "9203e1b7354e0b82fd5388ecc5ffe359"
server_key_iv: "5a4233fe79316513ddf2e559216e8814"
server_key_version: "encryptionkey@0"
I20260927 23:03:14.139017 25955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "8fe0073c69c04389ba0c9cf50a24c89e"
format_stamp: "Formatted at 2026-09-27 23:03:14 on dist-test-slave-5pd3"
server_key: "9203e1b7354e0b82fd5388ecc5ffe359"
server_key_iv: "5a4233fe79316513ddf2e559216e8814"
server_key_version: "encryptionkey@0"
I20260927 23:03:14.146639 25955 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260927 23:03:14.151968 25970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:14.154059 25955 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:03:14.154357 25955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "8fe0073c69c04389ba0c9cf50a24c89e"
format_stamp: "Formatted at 2026-09-27 23:03:14 on dist-test-slave-5pd3"
server_key: "9203e1b7354e0b82fd5388ecc5ffe359"
server_key_iv: "5a4233fe79316513ddf2e559216e8814"
server_key_version: "encryptionkey@0"
I20260927 23:03:14.154640 25955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:14.195107 25955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:14.201002 25955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:14.201450 25955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:14.226974 25955 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35711
I20260927 23:03:14.227017 26021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35711 every 8 connection(s)
I20260927 23:03:14.229014 25955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:14.232359 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 25955
I20260927 23:03:14.232621 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:14.233215 4394 external_mini_cluster.cc:1428] Setting key b829cb9d1f6421a8d779a2c6efd5c973
I20260927 23:03:14.234288 26022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:14.250838 26022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e: Bootstrap starting.
I20260927 23:03:14.252629 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:14.236622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:35793 (local address 127.0.0.1:35711)
0927 23:03:14.237763 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:14.237783 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:14.237844 (+ 61us) server_negotiation.cc:407] Connection header received
0927 23:03:14.240173 (+ 2329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:14.240211 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:14.240921 (+ 710us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:14.241464 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:14.242206 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:14.243890 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:14.246042 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:14.247049 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:14.247299 (+ 250us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:14.249030 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:14.249040 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:14.249092 (+ 52us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:14.249865 (+ 773us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:14.249989 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:14.250023 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:14.250196 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:14.250623 (+ 427us) server_negotiation.cc:299] Negotiation successful
0927 23:03:14.250890 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":475,"threads_started":1}
I20260927 23:03:14.254750 26022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:14.256167 26022 log.cc:824] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:14.258208 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:14.259907 26022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e: No bootstrap required, opened a new log
I20260927 23:03:14.270486 26022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe0073c69c04389ba0c9cf50a24c89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35711 } }
I20260927 23:03:14.271047 26022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:14.271257 26022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe0073c69c04389ba0c9cf50a24c89e, State: Initialized, Role: FOLLOWER
I20260927 23:03:14.272042 26022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe0073c69c04389ba0c9cf50a24c89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35711 } }
I20260927 23:03:14.272364 26022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:14.272593 26022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:14.272864 26022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:14.273033 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:14.264812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:36773 (local address 127.0.0.1:35711)
0927 23:03:14.265072 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:14.265081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:14.265552 (+ 471us) server_negotiation.cc:407] Connection header received
0927 23:03:14.265763 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:14.265769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:14.265883 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:14.266065 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:14.266718 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:14.268047 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:14.269589 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:14.270250 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:14.270436 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:14.271963 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:14.271969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:14.271974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:14.272001 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:14.272082 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:14.272108 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:14.272256 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:14.272543 (+ 287us) server_negotiation.cc:299] Negotiation successful
0927 23:03:14.272633 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260927 23:03:14.275910 26022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe0073c69c04389ba0c9cf50a24c89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35711 } }
I20260927 23:03:14.276491 26022 leader_election.cc:302] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fe0073c69c04389ba0c9cf50a24c89e; no voters:
I20260927 23:03:14.277194 26022 leader_election.cc:288] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:14.277472 26032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:14.278331 26032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [term 1 LEADER]: Becoming Leader. State: Replica: 8fe0073c69c04389ba0c9cf50a24c89e, State: Running, Role: LEADER
I20260927 23:03:14.279001 26032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe0073c69c04389ba0c9cf50a24c89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35711 } }
I20260927 23:03:14.279652 26022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:14.282416 26034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fe0073c69c04389ba0c9cf50a24c89e. Latest consensus state: current_term: 1 leader_uuid: "8fe0073c69c04389ba0c9cf50a24c89e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe0073c69c04389ba0c9cf50a24c89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35711 } } }
I20260927 23:03:14.283097 26033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fe0073c69c04389ba0c9cf50a24c89e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe0073c69c04389ba0c9cf50a24c89e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35711 } } }
I20260927 23:03:14.283512 26033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:14.284183 26034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:14.288551 26041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:14.294363 26041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:14.305424 26041 catalog_manager.cc:1347] Generated new cluster ID: bf097e7b9a1c479bb47f60d3af5a3cb5
I20260927 23:03:14.305612 26041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:14.332103 26041 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:14.334312 26041 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:14.350872 26041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fe0073c69c04389ba0c9cf50a24c89e: Generated new TSK 0
I20260927 23:03:14.352159 26041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:14.361572 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25955
W20260927 23:03:14.364001 26017 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.246:35793
W20260927 23:03:14.365113 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 25955: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:03:14.365304 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 25955
W20260927 23:03:14.386019 25940 connection.cc:535] client connection to 127.0.0.1:35711 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35711): Connection reset by peer (error 104)
2026-09-27T23:03:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2188 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:14 dist-test-slave-5pd3 krb5kdc[26053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:14 dist-test-slave-5pd3 krb5kdc[26053](info): set up 2 sockets
Sep 27 23:03:14 dist-test-slave-5pd3 krb5kdc[26053](info): commencing operation
krb5kdc: starting...
W20260927 23:03:15.524215 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:15 dist-test-slave-5pd3 krb5kdc[26053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 23:03:15.832567 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46513
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:16.191970 26066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:16.192303 26066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:16.192495 26066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:16.200170 26066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:16.200317 26066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:16.200367 26066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:16.200423 26066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:16.200475 26066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:16.211165 26066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46513
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:16.212738 26066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:16.214259 26066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:16.228500 26071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:16.228546 26074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:16.228608 26072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:16.230443 26066 server_base.cc:1034] running on GCE node
I20260927 23:03:16.231331 26066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:16.233064 26066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:16.234323 26066 hybrid_clock.cc:648] HybridClock initialized: now 1790550196234260 us; error 66 us; skew 500 ppm
I20260927 23:03:16.234721 26066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:16 dist-test-slave-5pd3 krb5kdc[26053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:16.238490 26066 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 23:03:16.240552 26066 webserver.cc:460] Webserver started at http://127.0.0.1:41197/ using document root <none> and password file <none>
I20260927 23:03:16.241348 26066 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:16.241490 26066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:16.241840 26066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:16.245244 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "93a3ba42e9714aec81465f8c0cece019"
format_stamp: "Formatted at 2026-09-27 23:03:16 on dist-test-slave-5pd3"
server_key: "95409de8df08169b81d7974934a1480a"
server_key_iv: "6718cb6f1a0fea8e6b85b7c108a1c399"
server_key_version: "encryptionkey@0"
I20260927 23:03:16.246121 26066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "93a3ba42e9714aec81465f8c0cece019"
format_stamp: "Formatted at 2026-09-27 23:03:16 on dist-test-slave-5pd3"
server_key: "95409de8df08169b81d7974934a1480a"
server_key_iv: "6718cb6f1a0fea8e6b85b7c108a1c399"
server_key_version: "encryptionkey@0"
I20260927 23:03:16.252575 26066 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260927 23:03:16.257053 26081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:16.258837 26066 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:03:16.259083 26066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "93a3ba42e9714aec81465f8c0cece019"
format_stamp: "Formatted at 2026-09-27 23:03:16 on dist-test-slave-5pd3"
server_key: "95409de8df08169b81d7974934a1480a"
server_key_iv: "6718cb6f1a0fea8e6b85b7c108a1c399"
server_key_version: "encryptionkey@0"
I20260927 23:03:16.259310 26066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:16.265977 26066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:16.270840 26066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:16.271176 26066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:16.293471 26066 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46513
I20260927 23:03:16.293473 26132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46513 every 8 connection(s)
I20260927 23:03:16.295267 26066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:16.300385 26133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:16.305528 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 26066
I20260927 23:03:16.305749 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:16.306406 4394 external_mini_cluster.cc:1428] Setting key bf6ab7c2f5223cb1abfdbd631e8b6220
I20260927 23:03:16.316609 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:16.308878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:47123 (local address 127.0.0.1:46513)
0927 23:03:16.309980 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:16.309996 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:16.310046 (+ 50us) server_negotiation.cc:407] Connection header received
0927 23:03:16.312162 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:16.312192 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:16.312690 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:16.313174 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:16.314636 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:16.314642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:16.314660 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:16.314685 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:16.314758 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:16.314782 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:16.315099 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:16.315251 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 23:03:16.315478 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":424,"threads_started":1}
I20260927 23:03:16.318284 26133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019: Bootstrap starting.
I20260927 23:03:16.321571 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:16.323000 26133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:16.324574 26133 log.cc:824] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:16.329303 26133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019: No bootstrap required, opened a new log
I20260927 23:03:16.330245 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:16.326753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.246:50633 (local address 127.0.0.1:46513)
0927 23:03:16.327020 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:16.327030 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:16.327713 (+ 683us) server_negotiation.cc:407] Connection header received
0927 23:03:16.327942 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:16.327947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:16.328018 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:16.328138 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:16.329313 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:16.329318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:16.329322 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:16.329350 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:16.329429 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:16.329456 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:16.329625 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:16.329804 (+ 179us) server_negotiation.cc:299] Negotiation successful
0927 23:03:16.329900 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260927 23:03:16.338392 26133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a3ba42e9714aec81465f8c0cece019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46513 } }
I20260927 23:03:16.338802 26133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:16.338950 26133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93a3ba42e9714aec81465f8c0cece019, State: Initialized, Role: FOLLOWER
I20260927 23:03:16.339624 26133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a3ba42e9714aec81465f8c0cece019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46513 } }
I20260927 23:03:16.339885 26133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:16.340057 26133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:16.340231 26133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:16.342880 26133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a3ba42e9714aec81465f8c0cece019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46513 } }
I20260927 23:03:16.343533 26133 leader_election.cc:302] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93a3ba42e9714aec81465f8c0cece019; no voters:
I20260927 23:03:16.344259 26133 leader_election.cc:288] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:16.344556 26143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:16.345999 26143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [term 1 LEADER]: Becoming Leader. State: Replica: 93a3ba42e9714aec81465f8c0cece019, State: Running, Role: LEADER
I20260927 23:03:16.346838 26133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:16.346858 26143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a3ba42e9714aec81465f8c0cece019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46513 } }
I20260927 23:03:16.351468 26145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93a3ba42e9714aec81465f8c0cece019. Latest consensus state: current_term: 1 leader_uuid: "93a3ba42e9714aec81465f8c0cece019" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a3ba42e9714aec81465f8c0cece019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46513 } } }
I20260927 23:03:16.351833 26144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93a3ba42e9714aec81465f8c0cece019" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a3ba42e9714aec81465f8c0cece019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46513 } } }
I20260927 23:03:16.352166 26145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:16.352262 26144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019 [sys.catalog]: This master's current role is: LEADER
W20260927 23:03:16.364316 26159 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:03:16.364547 26159 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:03:16.364761 26153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:16.367290 26153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:16.379155 26153 catalog_manager.cc:1347] Generated new cluster ID: 1565f8a3bd954ad6a5df57a1abdfa765
I20260927 23:03:16.379292 26153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:16.398984 26153 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:16.399694 26153 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:16.410879 26153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93a3ba42e9714aec81465f8c0cece019: Generated new TSK 0
I20260927 23:03:16.412199 26153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:16.425163 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26066
W20260927 23:03:16.427361 26128 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.246:47123
W20260927 23:03:16.428318 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26066: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:03:16.428556 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 26066
2026-09-27T23:03:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2062 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:16 dist-test-slave-5pd3 krb5kdc[26164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:16 dist-test-slave-5pd3 krb5kdc[26164](info): set up 2 sockets
Sep 27 23:03:16 dist-test-slave-5pd3 krb5kdc[26164](info): commencing operation
krb5kdc: starting...
W20260927 23:03:17.596693 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:17 dist-test-slave-5pd3 krb5kdc[26164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:03:17.911876 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:45483
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:43017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:45483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:18.342609 26177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:18.343034 26177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:18.343264 26177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:18.351679 26177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:18.351856 26177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:18.351984 26177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:18.352089 26177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:18.352190 26177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:18.363909 26177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:43017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:45483
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:45483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:18.365742 26177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:18.367564 26177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:18.382748 26185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:18.382793 26183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:18.383041 26182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:18.384017 26177 server_base.cc:1034] running on GCE node
I20260927 23:03:18.385210 26177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:18.387018 26177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:18.388293 26177 hybrid_clock.cc:648] HybridClock initialized: now 1790550198388250 us; error 56 us; skew 500 ppm
I20260927 23:03:18.388760 26177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:18 dist-test-slave-5pd3 krb5kdc[26164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550198, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:18.392472 26177 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:03:18.394412 26177 webserver.cc:460] Webserver started at http://127.4.74.190:34441/ using document root <none> and password file <none>
I20260927 23:03:18.395218 26177 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:18.395336 26177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:18.395716 26177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:18.399127 26177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "1809be7a9a2145f6a8683ff89345ca9e"
format_stamp: "Formatted at 2026-09-27 23:03:18 on dist-test-slave-5pd3"
server_key: "0e621b1eaacafac71e8193e7480faba8"
server_key_iv: "1379805bf41088a2722e72a93a3ac30a"
server_key_version: "encryptionkey@0"
I20260927 23:03:18.400074 26177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "1809be7a9a2145f6a8683ff89345ca9e"
format_stamp: "Formatted at 2026-09-27 23:03:18 on dist-test-slave-5pd3"
server_key: "0e621b1eaacafac71e8193e7480faba8"
server_key_iv: "1379805bf41088a2722e72a93a3ac30a"
server_key_version: "encryptionkey@0"
I20260927 23:03:18.406538 26177 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:03:18.411300 26192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:18.413269 26177 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260927 23:03:18.413559 26177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "1809be7a9a2145f6a8683ff89345ca9e"
format_stamp: "Formatted at 2026-09-27 23:03:18 on dist-test-slave-5pd3"
server_key: "0e621b1eaacafac71e8193e7480faba8"
server_key_iv: "1379805bf41088a2722e72a93a3ac30a"
server_key_version: "encryptionkey@0"
I20260927 23:03:18.413782 26177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:18.442924 26177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:18.448025 26177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:18.448410 26177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:18.472357 26177 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:45483
I20260927 23:03:18.472463 26243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:45483 every 8 connection(s)
I20260927 23:03:18.474014 26177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:18.475900 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 26177
I20260927 23:03:18.476135 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:18.476768 4394 external_mini_cluster.cc:1428] Setting key 2448313480e0d0ed34abb9cd62258182
I20260927 23:03:18.479982 26244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 23:03:18 dist-test-slave-5pd3 krb5kdc[26164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:03:18.496636 26244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e: Bootstrap starting.
I20260927 23:03:18.501636 26244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:18.503335 26244 log.cc:824] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:18.506387 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:18.478996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50645 (local address 127.4.74.190:45483)
0927 23:03:18.479940 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:18.479952 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:18.480006 (+ 54us) server_negotiation.cc:407] Connection header received
0927 23:03:18.482246 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:18.482278 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:18.482839 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:18.483297 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:18.485455 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:18.487111 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:18.489308 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:18.490169 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:18.490376 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:18.496308 (+ 5932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:18.496318 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:18.496340 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:18.496397 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:18.500163 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:18.501555 (+ 1392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:18.501564 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:18.501571 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:18.502064 (+ 493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:18.502737 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:18.502743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:18.502746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:18.503556 (+ 810us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:18.504075 (+ 519us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:18.504724 (+ 649us) server_negotiation.cc:299] Negotiation successful
0927 23:03:18.505163 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":418,"threads_started":1}
I20260927 23:03:18.507488 26244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e: No bootstrap required, opened a new log
I20260927 23:03:18.511868 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:18.519958 26244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1809be7a9a2145f6a8683ff89345ca9e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45483 } }
I20260927 23:03:18.520649 26244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:18.520785 26244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1809be7a9a2145f6a8683ff89345ca9e, State: Initialized, Role: FOLLOWER
I20260927 23:03:18.521579 26244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1809be7a9a2145f6a8683ff89345ca9e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45483 } }
I20260927 23:03:18.521850 26244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:18.522004 26244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:18.522202 26244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:18.525581 26244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1809be7a9a2145f6a8683ff89345ca9e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45483 } }
I20260927 23:03:18.526433 26244 leader_election.cc:302] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1809be7a9a2145f6a8683ff89345ca9e; no voters:
I20260927 23:03:18.527279 26244 leader_election.cc:288] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:18.527614 26254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:18.529247 26254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [term 1 LEADER]: Becoming Leader. State: Replica: 1809be7a9a2145f6a8683ff89345ca9e, State: Running, Role: LEADER
I20260927 23:03:18.530145 26254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1809be7a9a2145f6a8683ff89345ca9e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45483 } }
I20260927 23:03:18.530460 26244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:18.542819 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:18.518793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33209 (local address 127.4.74.190:45483)
0927 23:03:18.519062 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:18.519070 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:18.519737 (+ 667us) server_negotiation.cc:407] Connection header received
0927 23:03:18.519968 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:18.519974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:18.520099 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:18.520264 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:18.522408 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:18.523744 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:18.525212 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:18.525943 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:18.526122 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:18.530149 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:18.530159 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:18.530169 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:18.530228 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:18.536240 (+ 6012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:18.538045 (+ 1805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:18.538054 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:18.538067 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:18.538382 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:18.539133 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:18.539142 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:18.539148 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:18.539844 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:18.540467 (+ 623us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:18.540845 (+ 378us) server_negotiation.cc:299] Negotiation successful
0927 23:03:18.542116 (+ 1271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260927 23:03:18.547955 26256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1809be7a9a2145f6a8683ff89345ca9e. Latest consensus state: current_term: 1 leader_uuid: "1809be7a9a2145f6a8683ff89345ca9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1809be7a9a2145f6a8683ff89345ca9e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45483 } } }
I20260927 23:03:18.548477 26255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1809be7a9a2145f6a8683ff89345ca9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1809be7a9a2145f6a8683ff89345ca9e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 45483 } } }
I20260927 23:03:18.549243 26256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:18.549289 26255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e [sys.catalog]: This master's current role is: LEADER
W20260927 23:03:18.554025 26269 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:03:18.554193 26269 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:03:18.554360 26270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:18.556442 26270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:18.567270 26270 catalog_manager.cc:1347] Generated new cluster ID: c5756492c9954b09be7f3c46e00fd578
I20260927 23:03:18.567445 26270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:18.584372 26270 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:18.586153 26270 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:18.597045 26270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1809be7a9a2145f6a8683ff89345ca9e: Generated new TSK 0
I20260927 23:03:18.598294 26270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:18.612828 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26177
I20260927 23:03:18.614602 26239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:03:18.718645 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 26177
W20260927 23:03:18.738319 26162 connection.cc:535] client connection to 127.4.74.190:45483 recv error: Network error: failed to read from TLS socket (remote: 127.4.74.190:45483): Connection reset by peer (error 104)
2026-09-27T23:03:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:18 dist-test-slave-5pd3 krb5kdc[26276](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:18 dist-test-slave-5pd3 krb5kdc[26276](info): set up 2 sockets
Sep 27 23:03:18 dist-test-slave-5pd3 krb5kdc[26276](info): commencing operation
krb5kdc: starting...
W20260927 23:03:19.864322 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.005s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:20 dist-test-slave-5pd3 krb5kdc[26276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:03:20.192164 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:40587
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:40587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:20.637166 26289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:20.637698 26289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:20.638232 26289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:20.647516 26289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:20.647800 26289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:20.647909 26289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:20.648012 26289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:20.648111 26289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:20.660365 26289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:40587
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:40587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:20.662202 26289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:20.663942 26289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:20.678543 26295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:20.678632 26294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:20.678586 26297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:20.680665 26289 server_base.cc:1034] running on GCE node
I20260927 23:03:20.681747 26289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:20.683488 26289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:20.684830 26289 hybrid_clock.cc:648] HybridClock initialized: now 1790550200684768 us; error 57 us; skew 500 ppm
I20260927 23:03:20.685293 26289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:20 dist-test-slave-5pd3 krb5kdc[26276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550200, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:20.696133 26289 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:03:20.698082 26289 webserver.cc:460] Webserver started at http://127.4.74.190:39939/ using document root <none> and password file <none>
I20260927 23:03:20.698848 26289 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:20.698992 26289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:20.699272 26289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:20.702555 26289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "5e81e3a627ca4944bf6001382455052e"
format_stamp: "Formatted at 2026-09-27 23:03:20 on dist-test-slave-5pd3"
server_key: "842d7b1caba8c8e575d02c12574275ad"
server_key_iv: "22384ac11b7bd1de968991fea2f287eb"
server_key_version: "encryptionkey@0"
I20260927 23:03:20.703342 26289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "5e81e3a627ca4944bf6001382455052e"
format_stamp: "Formatted at 2026-09-27 23:03:20 on dist-test-slave-5pd3"
server_key: "842d7b1caba8c8e575d02c12574275ad"
server_key_iv: "22384ac11b7bd1de968991fea2f287eb"
server_key_version: "encryptionkey@0"
I20260927 23:03:20.711055 26289 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260927 23:03:20.716233 26304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:20.718550 26289 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.004s
I20260927 23:03:20.718901 26289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "5e81e3a627ca4944bf6001382455052e"
format_stamp: "Formatted at 2026-09-27 23:03:20 on dist-test-slave-5pd3"
server_key: "842d7b1caba8c8e575d02c12574275ad"
server_key_iv: "22384ac11b7bd1de968991fea2f287eb"
server_key_version: "encryptionkey@0"
I20260927 23:03:20.719250 26289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:20.740605 26289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:20.746016 26289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:20.746392 26289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:20.768968 26289 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:40587
I20260927 23:03:20.769084 26355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:40587 every 8 connection(s)
I20260927 23:03:20.770567 26289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:20.776495 26356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:20.778404 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 26289
I20260927 23:03:20.778643 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:20.779196 4394 external_mini_cluster.cc:1428] Setting key ae0751368182e2cf5ffa06387d685f87
I20260927 23:03:20.793725 26356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e: Bootstrap starting.
I20260927 23:03:20.795259 26359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:20.781531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41573 (local address 127.4.74.190:40587)
0927 23:03:20.782775 (+ 1244us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:20.782824 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:20.782882 (+ 58us) server_negotiation.cc:407] Connection header received
0927 23:03:20.784958 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:20.784991 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:20.785576 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:20.786080 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:20.786708 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:20.788173 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:20.789852 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:20.790703 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:20.790963 (+ 260us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:20.792607 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:20.792616 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:20.792633 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:20.793325 (+ 692us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:20.793432 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:20.793463 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:20.793655 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:20.793972 (+ 317us) server_negotiation.cc:299] Negotiation successful
0927 23:03:20.794179 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":891,"thread_start_us":552,"threads_started":1}
I20260927 23:03:20.798477 26356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:20.800025 26356 log.cc:824] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:20.800930 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:20.804400 26356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e: No bootstrap required, opened a new log
I20260927 23:03:20.815672 26359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:20.807305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33977 (local address 127.4.74.190:40587)
0927 23:03:20.807626 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:20.807642 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:20.808253 (+ 611us) server_negotiation.cc:407] Connection header received
0927 23:03:20.808551 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:20.808557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:20.808656 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:20.808841 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:20.809586 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:20.810471 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:20.812500 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:20.813086 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:20.813291 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:20.814715 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:20.814720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:20.814723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:20.814748 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:20.814819 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:20.814837 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:20.814970 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:20.815191 (+ 221us) server_negotiation.cc:299] Negotiation successful
0927 23:03:20.815264 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260927 23:03:20.815523 26356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e81e3a627ca4944bf6001382455052e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40587 } }
I20260927 23:03:20.816231 26356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:20.816390 26356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e81e3a627ca4944bf6001382455052e, State: Initialized, Role: FOLLOWER
I20260927 23:03:20.817154 26356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e81e3a627ca4944bf6001382455052e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40587 } }
I20260927 23:03:20.817474 26356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:20.817628 26356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:20.817824 26356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:20.820751 26356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e81e3a627ca4944bf6001382455052e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40587 } }
I20260927 23:03:20.821466 26356 leader_election.cc:302] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e81e3a627ca4944bf6001382455052e; no voters:
I20260927 23:03:20.822724 26356 leader_election.cc:288] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:20.823153 26366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:20.824268 26366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [term 1 LEADER]: Becoming Leader. State: Replica: 5e81e3a627ca4944bf6001382455052e, State: Running, Role: LEADER
I20260927 23:03:20.825205 26366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e81e3a627ca4944bf6001382455052e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40587 } }
I20260927 23:03:20.825881 26356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:20.828930 26367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e81e3a627ca4944bf6001382455052e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e81e3a627ca4944bf6001382455052e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40587 } } }
I20260927 23:03:20.829469 26367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:20.829058 26368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e81e3a627ca4944bf6001382455052e. Latest consensus state: current_term: 1 leader_uuid: "5e81e3a627ca4944bf6001382455052e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e81e3a627ca4944bf6001382455052e" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40587 } } }
I20260927 23:03:20.829644 26368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:20.832810 26373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:20.839743 26373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:20.850996 26373 catalog_manager.cc:1347] Generated new cluster ID: 50e74de795f44f61a433b8488ebfdc22
I20260927 23:03:20.851197 26373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:20.863102 26373 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:20.864943 26373 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:20.877960 26373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e81e3a627ca4944bf6001382455052e: Generated new TSK 0
I20260927 23:03:20.879130 26373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:20.890828 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26289
W20260927 23:03:20.893431 26351 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41573
W20260927 23:03:20.894454 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26289: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:03:20.894681 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 26289
W20260927 23:03:20.916859 26274 connection.cc:535] client connection to 127.4.74.190:40587 recv error: Network error: failed to read from TLS socket (remote: 127.4.74.190:40587): Connection reset by peer (error 104)
2026-09-27T23:03:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2178 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:21 dist-test-slave-5pd3 krb5kdc[26387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:21 dist-test-slave-5pd3 krb5kdc[26387](info): set up 2 sockets
Sep 27 23:03:21 dist-test-slave-5pd3 krb5kdc[26387](info): commencing operation
krb5kdc: starting...
W20260927 23:03:22.084501 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:22 dist-test-slave-5pd3 krb5kdc[26387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:03:22.408733 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:40391
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:40391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:22.826067 26400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:22.826409 26400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:22.826575 26400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:22.834200 26400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:22.834343 26400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:22.834391 26400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:22.834537 26400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:22.834613 26400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:22.845175 26400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:40391
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:40391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:22.846652 26400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:22.848212 26400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:22.861620 26405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:22.861590 26408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:22.861702 26406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:22.863749 26400 server_base.cc:1034] running on GCE node
I20260927 23:03:22.864660 26400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:22.866290 26400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:22.867658 26400 hybrid_clock.cc:648] HybridClock initialized: now 1790550202867563 us; error 83 us; skew 500 ppm
I20260927 23:03:22.868089 26400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:22 dist-test-slave-5pd3 krb5kdc[26387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550202, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:22.871855 26400 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:03:22.873667 26400 webserver.cc:460] Webserver started at http://127.4.74.190:33013/ using document root <none> and password file <none>
I20260927 23:03:22.874488 26400 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:22.874647 26400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:22.874981 26400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:22.878190 26400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "85ff304feeeb4f149c5db6b4bd0411f1"
format_stamp: "Formatted at 2026-09-27 23:03:22 on dist-test-slave-5pd3"
server_key: "c371146a0a911054cb7a581a9d5c53d5"
server_key_iv: "9a28ab9a3b40b04d8f564f24484e5504"
server_key_version: "encryptionkey@0"
I20260927 23:03:22.878994 26400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "85ff304feeeb4f149c5db6b4bd0411f1"
format_stamp: "Formatted at 2026-09-27 23:03:22 on dist-test-slave-5pd3"
server_key: "c371146a0a911054cb7a581a9d5c53d5"
server_key_iv: "9a28ab9a3b40b04d8f564f24484e5504"
server_key_version: "encryptionkey@0"
I20260927 23:03:22.885028 26400 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 23:03:22.889196 26415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:22.890931 26400 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260927 23:03:22.891165 26400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "85ff304feeeb4f149c5db6b4bd0411f1"
format_stamp: "Formatted at 2026-09-27 23:03:22 on dist-test-slave-5pd3"
server_key: "c371146a0a911054cb7a581a9d5c53d5"
server_key_iv: "9a28ab9a3b40b04d8f564f24484e5504"
server_key_version: "encryptionkey@0"
I20260927 23:03:22.891462 26400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:22.898065 26400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:22.902771 26400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:22.903098 26400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:22.926905 26400 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:40391
I20260927 23:03:22.927016 26466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:40391 every 8 connection(s)
I20260927 23:03:22.928648 26400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:22.933413 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 26400
I20260927 23:03:22.933640 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:22.934192 26467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:22.934262 4394 external_mini_cluster.cc:1428] Setting key e95b3e4020bb3a7ee1507230b77679ff
I20260927 23:03:22.944000 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:22.936573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56121 (local address 127.4.74.190:40391)
0927 23:03:22.937514 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:22.937526 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:22.937583 (+ 57us) server_negotiation.cc:407] Connection header received
0927 23:03:22.939961 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:22.939986 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:22.940425 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:22.940807 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:22.942184 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:22.942191 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:22.942203 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:22.942219 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:22.942284 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:22.942311 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:22.942517 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:22.942617 (+ 100us) server_negotiation.cc:299] Negotiation successful
0927 23:03:22.942858 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":432,"threads_started":1}
I20260927 23:03:22.948184 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:22.952236 26467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1: Bootstrap starting.
I20260927 23:03:22.958447 26467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:22.960170 26467 log.cc:824] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:22.961303 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:22.958131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52627 (local address 127.4.74.190:40391)
0927 23:03:22.958399 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:22.958408 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:22.958909 (+ 501us) server_negotiation.cc:407] Connection header received
0927 23:03:22.959083 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:22.959092 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:22.959188 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:22.959360 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:22.960478 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:22.960483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:22.960487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:22.960509 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:22.960572 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:22.960591 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:22.960733 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:22.960853 (+ 120us) server_negotiation.cc:299] Negotiation successful
0927 23:03:22.960930 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260927 23:03:22.964048 26467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1: No bootstrap required, opened a new log
I20260927 23:03:22.972729 26467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ff304feeeb4f149c5db6b4bd0411f1" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40391 } }
I20260927 23:03:22.973385 26467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:22.973537 26467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85ff304feeeb4f149c5db6b4bd0411f1, State: Initialized, Role: FOLLOWER
I20260927 23:03:22.974262 26467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ff304feeeb4f149c5db6b4bd0411f1" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40391 } }
I20260927 23:03:22.974547 26467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:22.974715 26467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:22.974918 26467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:22.977658 26467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ff304feeeb4f149c5db6b4bd0411f1" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40391 } }
I20260927 23:03:22.978698 26467 leader_election.cc:302] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85ff304feeeb4f149c5db6b4bd0411f1; no voters:
I20260927 23:03:22.979636 26467 leader_election.cc:288] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:22.979888 26477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:22.980787 26477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [term 1 LEADER]: Becoming Leader. State: Replica: 85ff304feeeb4f149c5db6b4bd0411f1, State: Running, Role: LEADER
I20260927 23:03:22.981529 26477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ff304feeeb4f149c5db6b4bd0411f1" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40391 } }
I20260927 23:03:22.983035 26467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:22.984913 26479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85ff304feeeb4f149c5db6b4bd0411f1. Latest consensus state: current_term: 1 leader_uuid: "85ff304feeeb4f149c5db6b4bd0411f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ff304feeeb4f149c5db6b4bd0411f1" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40391 } } }
I20260927 23:03:22.985438 26479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:22.987991 26478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85ff304feeeb4f149c5db6b4bd0411f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ff304feeeb4f149c5db6b4bd0411f1" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40391 } } }
I20260927 23:03:22.988771 26478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:22.991338 26482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:22.997504 26482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:23.009384 26482 catalog_manager.cc:1347] Generated new cluster ID: ff6f06a98e7d4ec0860747f2ed3de349
I20260927 23:03:23.009644 26482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:23.027725 26482 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:23.028334 26482 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:23.040369 26482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85ff304feeeb4f149c5db6b4bd0411f1: Generated new TSK 0
I20260927 23:03:23.041661 26482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:23.054507 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26400
W20260927 23:03:23.057595 26462 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56121
W20260927 23:03:23.058576 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26400: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:03:23.058806 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 26400
2026-09-27T23:03:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2162 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:23 dist-test-slave-5pd3 krb5kdc[26498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:23 dist-test-slave-5pd3 krb5kdc[26498](info): set up 2 sockets
Sep 27 23:03:23 dist-test-slave-5pd3 krb5kdc[26498](info): commencing operation
krb5kdc: starting...
W20260927 23:03:24.192180 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.012s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:24 dist-test-slave-5pd3 krb5kdc[26498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:03:24.510978 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:40961
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:38155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:40961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:24.917994 26511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:24.918326 26511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:24.918488 26511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:24.926798 26511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:24.926942 26511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:24.926991 26511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:24.927101 26511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:24.927165 26511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:24.937997 26511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:38155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:40961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:40961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:24.939484 26511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:24.941021 26511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:24.956156 26519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:24.956269 26516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:24.956269 26517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:24.957147 26511 server_base.cc:1034] running on GCE node
I20260927 23:03:24.958039 26511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:24.959852 26511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:24.961128 26511 hybrid_clock.cc:648] HybridClock initialized: now 1790550204961080 us; error 52 us; skew 500 ppm
I20260927 23:03:24.961588 26511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:24 dist-test-slave-5pd3 krb5kdc[26498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550204, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:24.965549 26511 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:03:24.967482 26511 webserver.cc:460] Webserver started at http://127.4.74.190:42147/ using document root <none> and password file <none>
I20260927 23:03:24.968288 26511 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:24.968441 26511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:24.968782 26511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:24.972036 26511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "166a830188e24888ab5a43d5be0c45dc"
format_stamp: "Formatted at 2026-09-27 23:03:24 on dist-test-slave-5pd3"
server_key: "3a1369884165b3d364b67f479bf77f1e"
server_key_iv: "bbd6b9a9c6821ab83f887fe6cb883ced"
server_key_version: "encryptionkey@0"
I20260927 23:03:24.972862 26511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "166a830188e24888ab5a43d5be0c45dc"
format_stamp: "Formatted at 2026-09-27 23:03:24 on dist-test-slave-5pd3"
server_key: "3a1369884165b3d364b67f479bf77f1e"
server_key_iv: "bbd6b9a9c6821ab83f887fe6cb883ced"
server_key_version: "encryptionkey@0"
I20260927 23:03:24.979274 26511 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260927 23:03:24.983704 26526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:24.985396 26511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:03:24.985613 26511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "166a830188e24888ab5a43d5be0c45dc"
format_stamp: "Formatted at 2026-09-27 23:03:24 on dist-test-slave-5pd3"
server_key: "3a1369884165b3d364b67f479bf77f1e"
server_key_iv: "bbd6b9a9c6821ab83f887fe6cb883ced"
server_key_version: "encryptionkey@0"
I20260927 23:03:24.985833 26511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:25.004964 26511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:25.009699 26511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:25.010018 26511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:25.041317 26511 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:40961
I20260927 23:03:25.041401 26577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:40961 every 8 connection(s)
I20260927 23:03:25.043172 26511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:25.045271 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 26511
I20260927 23:03:25.045504 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:25.046137 4394 external_mini_cluster.cc:1428] Setting key 103943a26b4f99f94e9c556db1dd5534
I20260927 23:03:25.050115 26578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 27 23:03:25 dist-test-slave-5pd3 krb5kdc[26498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:03:25.068189 26578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc: Bootstrap starting.
I20260927 23:03:25.072979 26578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:25.074496 26578 log.cc:824] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:25.078114 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:25.049067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59559 (local address 127.4.74.190:40961)
0927 23:03:25.050724 (+ 1657us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:25.050742 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:25.050849 (+ 107us) server_negotiation.cc:407] Connection header received
0927 23:03:25.052846 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:25.052875 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:25.053394 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:25.053889 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:25.056351 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:25.058637 (+ 2286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:25.060985 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:25.061855 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:25.062041 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:25.068362 (+ 6321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:25.068380 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:25.068407 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:25.068479 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:25.072537 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:25.073664 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:25.073672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:25.073680 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:25.073899 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:25.074329 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:25.074333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:25.074335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:25.075059 (+ 724us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:25.075414 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:25.075998 (+ 584us) server_negotiation.cc:299] Negotiation successful
0927 23:03:25.076602 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":984,"thread_start_us":514,"threads_started":1}
I20260927 23:03:25.078984 26578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc: No bootstrap required, opened a new log
I20260927 23:03:25.083274 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:25.087903 26578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166a830188e24888ab5a43d5be0c45dc" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40961 } }
I20260927 23:03:25.088366 26578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:25.088474 26578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166a830188e24888ab5a43d5be0c45dc, State: Initialized, Role: FOLLOWER
I20260927 23:03:25.089236 26578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166a830188e24888ab5a43d5be0c45dc" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40961 } }
I20260927 23:03:25.089582 26578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:25.089766 26578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:25.089967 26578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:25.093616 26578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166a830188e24888ab5a43d5be0c45dc" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40961 } }
I20260927 23:03:25.094574 26578 leader_election.cc:302] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 166a830188e24888ab5a43d5be0c45dc; no voters:
I20260927 23:03:25.095467 26578 leader_election.cc:288] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:25.095744 26588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:25.098117 26578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:25.103175 26588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [term 1 LEADER]: Becoming Leader. State: Replica: 166a830188e24888ab5a43d5be0c45dc, State: Running, Role: LEADER
I20260927 23:03:25.104197 26588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166a830188e24888ab5a43d5be0c45dc" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40961 } }
I20260927 23:03:25.112167 26589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "166a830188e24888ab5a43d5be0c45dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166a830188e24888ab5a43d5be0c45dc" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40961 } } }
I20260927 23:03:25.112792 26589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:25.113020 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:25.089130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34281 (local address 127.4.74.190:40961)
0927 23:03:25.089417 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:25.089428 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:25.090420 (+ 992us) server_negotiation.cc:407] Connection header received
0927 23:03:25.090641 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:25.090646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:25.090744 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:25.090876 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:25.093031 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:25.094080 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:25.096504 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:25.097257 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:25.097384 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:25.101716 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:25.101730 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:25.101739 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:25.101824 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:25.106785 (+ 4961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:25.108369 (+ 1584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:25.108378 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:25.108389 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:25.108660 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:25.109307 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:25.109319 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:25.109325 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:25.110029 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:25.110388 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:25.112057 (+ 1669us) server_negotiation.cc:299] Negotiation successful
0927 23:03:25.112445 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260927 23:03:25.115978 26590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 166a830188e24888ab5a43d5be0c45dc. Latest consensus state: current_term: 1 leader_uuid: "166a830188e24888ab5a43d5be0c45dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "166a830188e24888ab5a43d5be0c45dc" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 40961 } } }
I20260927 23:03:25.117162 26590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc [sys.catalog]: This master's current role is: LEADER
W20260927 23:03:25.122182 26602 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:03:25.122494 26602 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:03:25.125387 26603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:25.127852 26603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:25.139701 26603 catalog_manager.cc:1347] Generated new cluster ID: a2556971b98a45b398443746511cdbb8
I20260927 23:03:25.139869 26603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:25.165585 26603 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:25.167284 26603 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:25.178000 26603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 166a830188e24888ab5a43d5be0c45dc: Generated new TSK 0
I20260927 23:03:25.179332 26603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:25.191576 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26511
I20260927 23:03:25.193529 26573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:03:25.291124 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 26511
2026-09-27T23:03:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2226 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:25 dist-test-slave-5pd3 krb5kdc[26611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:25 dist-test-slave-5pd3 krb5kdc[26611](info): set up 2 sockets
Sep 27 23:03:25 dist-test-slave-5pd3 krb5kdc[26611](info): commencing operation
krb5kdc: starting...
W20260927 23:03:26.452371 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:26 dist-test-slave-5pd3 krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:03:26.764263 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:37067
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:41837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:37067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:27.139314 26624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:27.139732 26624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:27.139842 26624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:27.148080 26624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:27.148213 26624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:27.148260 26624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:27.148319 26624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:27.148367 26624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:27.158550 26624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:41837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:37067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:37067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:27.159996 26624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:27.161321 26624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:27.174568 26629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:27.174674 26630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:27.174583 26632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:27.175951 26624 server_base.cc:1034] running on GCE node
I20260927 23:03:27.176918 26624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:27.178514 26624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:27.179805 26624 hybrid_clock.cc:648] HybridClock initialized: now 1790550207179727 us; error 63 us; skew 500 ppm
I20260927 23:03:27.180236 26624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:27 dist-test-slave-5pd3 krb5kdc[26611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550207, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:27.190050 26624 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:03:27.192097 26624 webserver.cc:460] Webserver started at http://127.4.74.190:46477/ using document root <none> and password file <none>
I20260927 23:03:27.192863 26624 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:27.192973 26624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:27.193248 26624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:27.196853 26624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "87839604015a4984be7847bc431b95ef"
format_stamp: "Formatted at 2026-09-27 23:03:27 on dist-test-slave-5pd3"
server_key: "6b767d92ec385f659324ca0371607d82"
server_key_iv: "3550d3f499fc5df68338918c6646a47a"
server_key_version: "encryptionkey@0"
I20260927 23:03:27.197659 26624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "87839604015a4984be7847bc431b95ef"
format_stamp: "Formatted at 2026-09-27 23:03:27 on dist-test-slave-5pd3"
server_key: "6b767d92ec385f659324ca0371607d82"
server_key_iv: "3550d3f499fc5df68338918c6646a47a"
server_key_version: "encryptionkey@0"
I20260927 23:03:27.203871 26624 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260927 23:03:27.208509 26639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:27.210371 26624 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:03:27.210575 26624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "87839604015a4984be7847bc431b95ef"
format_stamp: "Formatted at 2026-09-27 23:03:27 on dist-test-slave-5pd3"
server_key: "6b767d92ec385f659324ca0371607d82"
server_key_iv: "3550d3f499fc5df68338918c6646a47a"
server_key_version: "encryptionkey@0"
I20260927 23:03:27.210791 26624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:27.239423 26624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:27.244115 26624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:27.244392 26624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:27.266336 26624 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:37067
I20260927 23:03:27.266400 26690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:37067 every 8 connection(s)
I20260927 23:03:27.267977 26624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:27.272761 26691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:27.277730 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 26624
I20260927 23:03:27.277899 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:27.278447 4394 external_mini_cluster.cc:1428] Setting key 415c57b8c612754fb90ee0295b4a57a8
I20260927 23:03:27.288215 26691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef: Bootstrap starting.
I20260927 23:03:27.292434 26691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:27.293982 26691 log.cc:824] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:27.294118 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:27.280745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49043 (local address 127.4.74.190:37067)
0927 23:03:27.281811 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:27.281827 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:27.281889 (+ 62us) server_negotiation.cc:407] Connection header received
0927 23:03:27.283964 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:27.283993 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:27.284556 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:27.285043 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:27.285717 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:27.287222 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:27.288854 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:27.289639 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:27.289890 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:27.291326 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:27.291335 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:27.291358 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:27.292091 (+ 733us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:27.292191 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:27.292219 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:27.292372 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:27.292628 (+ 256us) server_negotiation.cc:299] Negotiation successful
0927 23:03:27.292963 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":477,"threads_started":1}
I20260927 23:03:27.298336 26691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef: No bootstrap required, opened a new log
I20260927 23:03:27.299438 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:27.308766 26691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87839604015a4984be7847bc431b95ef" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37067 } }
I20260927 23:03:27.309450 26691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:27.309582 26691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87839604015a4984be7847bc431b95ef, State: Initialized, Role: FOLLOWER
I20260927 23:03:27.310453 26691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87839604015a4984be7847bc431b95ef" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37067 } }
I20260927 23:03:27.310810 26691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:27.311012 26691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:27.311254 26691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:27.312382 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:27.304568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56113 (local address 127.4.74.190:37067)
0927 23:03:27.304830 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:27.304837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:27.305143 (+ 306us) server_negotiation.cc:407] Connection header received
0927 23:03:27.305335 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:27.305342 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:27.305459 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:27.305617 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:27.306192 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:27.307428 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:27.309004 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:27.309751 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:27.309901 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:27.311417 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:27.311427 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:27.311434 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:27.311465 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:27.311524 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:27.311587 (+ 63us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:27.311704 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:27.311871 (+ 167us) server_negotiation.cc:299] Negotiation successful
0927 23:03:27.312006 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260927 23:03:27.314620 26691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87839604015a4984be7847bc431b95ef" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37067 } }
I20260927 23:03:27.315376 26691 leader_election.cc:302] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87839604015a4984be7847bc431b95ef; no voters:
I20260927 23:03:27.316092 26691 leader_election.cc:288] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:27.316282 26701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:27.317276 26701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [term 1 LEADER]: Becoming Leader. State: Replica: 87839604015a4984be7847bc431b95ef, State: Running, Role: LEADER
I20260927 23:03:27.317974 26701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87839604015a4984be7847bc431b95ef" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37067 } }
I20260927 23:03:27.318903 26691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:27.321448 26703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87839604015a4984be7847bc431b95ef. Latest consensus state: current_term: 1 leader_uuid: "87839604015a4984be7847bc431b95ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87839604015a4984be7847bc431b95ef" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37067 } } }
I20260927 23:03:27.322042 26703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:27.323524 26707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:27.323439 26702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87839604015a4984be7847bc431b95ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87839604015a4984be7847bc431b95ef" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37067 } } }
I20260927 23:03:27.324013 26702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:27.331107 26707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:27.342628 26707 catalog_manager.cc:1347] Generated new cluster ID: 508a678bee15465698e97e3d12c0f94d
I20260927 23:03:27.342795 26707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:27.371021 26707 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:27.373176 26707 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:27.383518 26707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87839604015a4984be7847bc431b95ef: Generated new TSK 0
I20260927 23:03:27.384785 26707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:27.402712 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26624
W20260927 23:03:27.404275 26686 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49043
W20260927 23:03:27.405210 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26624: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:03:27.405428 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 26624
2026-09-27T23:03:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2120 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:27 dist-test-slave-5pd3 krb5kdc[26722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:27 dist-test-slave-5pd3 krb5kdc[26722](info): set up 2 sockets
Sep 27 23:03:27 dist-test-slave-5pd3 krb5kdc[26722](info): commencing operation
krb5kdc: starting...
W20260927 23:03:28.576735 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:28 dist-test-slave-5pd3 krb5kdc[26722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:03:28.900326 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36459
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:39767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:29.259168 26735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:29.259502 26735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:29.259685 26735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:29.267307 26735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:29.267452 26735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:29.267503 26735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:29.267630 26735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:29.267704 26735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:29.278123 26735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:39767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:29.279410 26735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:29.280763 26735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:29.294322 26740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:29.294498 26741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:29.294449 26743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:29.296049 26735 server_base.cc:1034] running on GCE node
I20260927 23:03:29.296959 26735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:29.298578 26735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:29.300027 26735 hybrid_clock.cc:648] HybridClock initialized: now 1790550209300019 us; error 150 us; skew 500 ppm
I20260927 23:03:29.300453 26735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:29 dist-test-slave-5pd3 krb5kdc[26722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550209, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:29.304409 26735 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:03:29.306351 26735 webserver.cc:460] Webserver started at http://127.4.74.190:43577/ using document root <none> and password file <none>
I20260927 23:03:29.307107 26735 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:29.307265 26735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:29.307662 26735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:29.310757 26735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd"
format_stamp: "Formatted at 2026-09-27 23:03:29 on dist-test-slave-5pd3"
server_key: "946c59af7ef18672b12720844f66e9c5"
server_key_iv: "0edc040012ef8f606bb0261fd4abbcc4"
server_key_version: "encryptionkey@0"
I20260927 23:03:29.311619 26735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd"
format_stamp: "Formatted at 2026-09-27 23:03:29 on dist-test-slave-5pd3"
server_key: "946c59af7ef18672b12720844f66e9c5"
server_key_iv: "0edc040012ef8f606bb0261fd4abbcc4"
server_key_version: "encryptionkey@0"
I20260927 23:03:29.317773 26735 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260927 23:03:29.322660 26750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:29.324545 26735 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260927 23:03:29.324805 26735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd"
format_stamp: "Formatted at 2026-09-27 23:03:29 on dist-test-slave-5pd3"
server_key: "946c59af7ef18672b12720844f66e9c5"
server_key_iv: "0edc040012ef8f606bb0261fd4abbcc4"
server_key_version: "encryptionkey@0"
I20260927 23:03:29.325044 26735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:29.331933 26735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:29.336833 26735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:29.337265 26735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:29.360009 26735 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:36459
I20260927 23:03:29.360122 26801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:36459 every 8 connection(s)
I20260927 23:03:29.361624 26735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:29.363612 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 26735
I20260927 23:03:29.363837 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:29.364446 4394 external_mini_cluster.cc:1428] Setting key be46738554dbac589b0d0aae654cc3ef
I20260927 23:03:29.367762 26802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:29.374076 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:29.366756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55697 (local address 127.4.74.190:36459)
0927 23:03:29.367933 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:29.367955 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:29.368016 (+ 61us) server_negotiation.cc:407] Connection header received
0927 23:03:29.369886 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:29.369918 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:29.370367 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:29.370868 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:29.372166 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:29.372173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:29.372186 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:29.372209 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:29.372308 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:29.372333 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:29.372634 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:29.372790 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 23:03:29.373007 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":442,"threads_started":1}
I20260927 23:03:29.379375 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:29.382599 26802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd: Bootstrap starting.
I20260927 23:03:29.387156 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:29.384166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49941 (local address 127.4.74.190:36459)
0927 23:03:29.384417 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:29.384423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:29.384884 (+ 461us) server_negotiation.cc:407] Connection header received
0927 23:03:29.385131 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:29.385135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:29.385197 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:29.385306 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:29.386405 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:29.386410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:29.386413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 23:03:29.386429 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:29.386473 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 23:03:29.386489 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:29.386593 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:29.386737 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 23:03:29.386803 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260927 23:03:29.387650 26802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:29.389211 26802 log.cc:824] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:29.392863 26802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd: No bootstrap required, opened a new log
I20260927 23:03:29.401453 26802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36459 } }
I20260927 23:03:29.401983 26802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:03:29.402127 26802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dfac05ca52d4dddbdd20d3bfa4266bd, State: Initialized, Role: FOLLOWER
I20260927 23:03:29.402853 26802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36459 } }
I20260927 23:03:29.403136 26802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 23:03:29.403313 26802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 23:03:29.403529 26802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:29.406194 26802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36459 } }
I20260927 23:03:29.407193 26802 leader_election.cc:302] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dfac05ca52d4dddbdd20d3bfa4266bd; no voters:
I20260927 23:03:29.408147 26802 leader_election.cc:288] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 23:03:29.408368 26812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:29.409381 26812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [term 1 LEADER]: Becoming Leader. State: Replica: 2dfac05ca52d4dddbdd20d3bfa4266bd, State: Running, Role: LEADER
I20260927 23:03:29.410108 26812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36459 } }
I20260927 23:03:29.410944 26802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:29.413463 26814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dfac05ca52d4dddbdd20d3bfa4266bd. Latest consensus state: current_term: 1 leader_uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36459 } } }
I20260927 23:03:29.413967 26814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:29.415261 26818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:29.415067 26813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfac05ca52d4dddbdd20d3bfa4266bd" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36459 } } }
I20260927 23:03:29.415634 26813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:29.422858 26818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:29.435603 26818 catalog_manager.cc:1347] Generated new cluster ID: 087b8fdb12044e41a55109c9c196ddb5
I20260927 23:03:29.435827 26818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:29.484591 26818 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:29.485239 26818 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:29.500183 26818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dfac05ca52d4dddbdd20d3bfa4266bd: Generated new TSK 0
I20260927 23:03:29.501446 26818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:29.519371 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26735
W20260927 23:03:29.521513 26797 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55697
W20260927 23:03:29.522478 4394 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26735: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260927 23:03:29.522733 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 26735
2026-09-27T23:03:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2115 ms)
[----------] 18 tests from AuthTokenIssuingTest (39325 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:29 dist-test-slave-5pd3 krb5kdc[26833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:29 dist-test-slave-5pd3 krb5kdc[26833](info): set up 2 sockets
Sep 27 23:03:29 dist-test-slave-5pd3 krb5kdc[26833](info): commencing operation
krb5kdc: starting...
W20260927 23:03:30.672319 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.008s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:30 dist-test-slave-5pd3 krb5kdc[26833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:03:30.988992 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:37757
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:37757,127.4.74.189:43853,127.4.74.188:45325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:31.346513 26846 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 23:03:31.347215 26846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:31.347345 26846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:31.347506 26846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:31.358448 26846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:31.358587 26846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:31.358685 26846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:31.358781 26846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:31.358872 26846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:31.373616 26846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:37757,127.4.74.189:43853,127.4.74.188:45325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:37757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:31.374995 26846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:31.376353 26846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:31.392273 26854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:31.392225 26852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:31.393314 26851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:31.394936 26846 server_base.cc:1034] running on GCE node
I20260927 23:03:31.395931 26846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:31.397496 26846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:31.398983 26846 hybrid_clock.cc:648] HybridClock initialized: now 1790550211398827 us; error 116 us; skew 500 ppm
I20260927 23:03:31.399343 26846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:31 dist-test-slave-5pd3 krb5kdc[26833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550211, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:31.403000 26846 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:03:31.404871 26846 webserver.cc:460] Webserver started at http://127.4.74.190:46607/ using document root <none> and password file <none>
I20260927 23:03:31.405604 26846 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:31.405731 26846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:31.406075 26846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:31.409197 26846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "34f227b94537444cb1c0f569b3a3c2d2"
format_stamp: "Formatted at 2026-09-27 23:03:31 on dist-test-slave-5pd3"
server_key: "8b8a206bd17739711826f87472a0f6c6"
server_key_iv: "4cd43233843b52cf64e78358ec978f8b"
server_key_version: "encryptionkey@0"
I20260927 23:03:31.410005 26846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "34f227b94537444cb1c0f569b3a3c2d2"
format_stamp: "Formatted at 2026-09-27 23:03:31 on dist-test-slave-5pd3"
server_key: "8b8a206bd17739711826f87472a0f6c6"
server_key_iv: "4cd43233843b52cf64e78358ec978f8b"
server_key_version: "encryptionkey@0"
I20260927 23:03:31.415663 26846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260927 23:03:31.419528 26861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:31.421270 26846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260927 23:03:31.421504 26846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "34f227b94537444cb1c0f569b3a3c2d2"
format_stamp: "Formatted at 2026-09-27 23:03:31 on dist-test-slave-5pd3"
server_key: "8b8a206bd17739711826f87472a0f6c6"
server_key_iv: "4cd43233843b52cf64e78358ec978f8b"
server_key_version: "encryptionkey@0"
I20260927 23:03:31.421757 26846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:31.439637 26846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:31.444573 26846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:31.444897 26846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:31.467186 26846 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:37757
I20260927 23:03:31.467295 26912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:37757 every 8 connection(s)
I20260927 23:03:31.468771 26846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:31.473704 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 26846
I20260927 23:03:31.473989 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:31.474051 26913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:31.474956 4394 external_mini_cluster.cc:1428] Setting key a1a00a41fb5d135b320cd25e588adcec
I20260927 23:03:31.486292 26913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:03:31 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550211, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.74.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.189@KRBTEST.COM" created.
I20260927 23:03:31.517170 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.491353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34588 (local address 127.4.74.190:37757)
0927 23:03:31.493763 (+ 2410us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:31.493770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:31.493827 (+ 57us) server_negotiation.cc:407] Connection header received
0927 23:03:31.496369 (+ 2542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:31.496376 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:31.496718 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:31.496944 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:31.498903 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:31.500393 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:31.502821 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:31.503492 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:31.503767 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:31.508104 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:31.508114 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:31.508131 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:31.508171 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:31.511941 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:31.512854 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:31.512864 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:31.512870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:31.513055 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:31.513458 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:31.513466 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:31.513469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:31.514120 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:31.514353 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:31.514959 (+ 606us) server_negotiation.cc:299] Negotiation successful
0927 23:03:31.515393 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":2181,"spinlock_wait_cycles":56576,"thread_start_us":1865,"threads_started":1}
I20260927 23:03:31.517197 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.491353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37757 (local address 127.0.0.1:34588)
0927 23:03:31.492576 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.492641 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:31.495846 (+ 3205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:31.497062 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:31.497074 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:31.497990 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:31.498582 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:31.498612 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:31.500604 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:31.500614 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:31.502628 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:31.502649 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:31.503880 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:31.503892 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:31.504153 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:31.505107 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:31.505136 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:31.507828 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:31.512182 (+ 4354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:31.512196 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:31.512217 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:31.512703 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:31.513189 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:31.513194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:31.513196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:31.513350 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:31.514283 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:31.514292 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:31.514564 (+ 272us) client_negotiation.cc:769] Sending connection context
0927 23:03:31.514849 (+ 285us) client_negotiation.cc:240] Negotiation successful
0927 23:03:31.515527 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":161,"thread_start_us":507,"threads_started":1}
I20260927 23:03:31.521111 26913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:03:31.522644 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.522122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:39948)
0927 23:03:31.522338 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.522482 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260927 23:03:31.522989 26865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
I20260927 23:03:31.525012 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.524542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:39964)
0927 23:03:31.524706 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.524788 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260927 23:03:31.525326 26913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:43853: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab.
Entry for principal kudu/127.4.74.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab.
I20260927 23:03:31.548651 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.189:43853
--webserver_interface=127.4.74.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab
--principal=kudu/127.4.74.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:37757,127.4.74.189:43853,127.4.74.188:45325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:03:31.574784 26913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } attempt: 1
I20260927 23:03:31.576391 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.575846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:39970)
0927 23:03:31.576095 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.576224 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260927 23:03:31.577484 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.577100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:39978)
0927 23:03:31.577257 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.577327 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260927 23:03:31.577715 26913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:43853: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
I20260927 23:03:31.646097 26913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } attempt: 2
I20260927 23:03:31.647716 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.647029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:39990)
0927 23:03:31.647350 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.647454 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260927 23:03:31.648934 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.648516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:40004)
0927 23:03:31.648686 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.648770 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260927 23:03:31.649272 26913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:43853: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
I20260927 23:03:31.740624 26913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } attempt: 3
I20260927 23:03:31.742379 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.741697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:40014)
0927 23:03:31.742054 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.742186 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260927 23:03:31.743680 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.743172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:40022)
0927 23:03:31.743327 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.743414 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260927 23:03:31.743866 26913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:43853: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
I20260927 23:03:31.887133 26913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } attempt: 4
I20260927 23:03:31.888741 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.888106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:40038)
0927 23:03:31.888462 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.888576 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260927 23:03:31.890487 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:31.889512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:40050)
0927 23:03:31.889674 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:31.890304 (+ 630us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260927 23:03:31.890751 26913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:43853: Network error: Client connection negotiation failed: client connection to 127.4.74.189:43853: connect: Connection refused (error 111)
W20260927 23:03:31.905809 26920 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 23:03:31.906205 26920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:31.906347 26920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:31.906482 26920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:31.914237 26920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:31.914355 26920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:31.914460 26920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:31.914549 26920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:31.914671 26920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:31.926267 26920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:37757,127.4.74.189:43853,127.4.74.188:45325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab
--principal=kudu/127.4.74.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.189:43853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.74.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:31.927776 26920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:31.929191 26920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:31.942667 26925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:31.942842 26926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:31.942870 26928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:31.944993 26920 server_base.cc:1034] running on GCE node
I20260927 23:03:31.946455 26920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:31.948764 26920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:31.950150 26920 hybrid_clock.cc:648] HybridClock initialized: now 1790550211950031 us; error 84 us; skew 500 ppm
I20260927 23:03:31.950680 26920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:31 dist-test-slave-5pd3 krb5kdc[26833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550211, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:31.955026 26920 init.cc:377] Logged in from keytab as kudu/127.4.74.189@KRBTEST.COM (short username kudu)
I20260927 23:03:31.957181 26920 webserver.cc:460] Webserver started at http://127.4.74.189:32865/ using document root <none> and password file <none>
I20260927 23:03:31.958320 26920 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:31.958451 26920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:31.958891 26920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:31.962975 26920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/data/instance:
uuid: "b6bc59fe31e2493da1adca15be811cc1"
format_stamp: "Formatted at 2026-09-27 23:03:31 on dist-test-slave-5pd3"
server_key: "d53c10f4614e37bdc2e56fc875ec96fc"
server_key_iv: "545064d1181380c4411555fdeebeb4d0"
server_key_version: "encryptionkey@0"
I20260927 23:03:31.964046 26920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/wal/instance:
uuid: "b6bc59fe31e2493da1adca15be811cc1"
format_stamp: "Formatted at 2026-09-27 23:03:31 on dist-test-slave-5pd3"
server_key: "d53c10f4614e37bdc2e56fc875ec96fc"
server_key_iv: "545064d1181380c4411555fdeebeb4d0"
server_key_version: "encryptionkey@0"
I20260927 23:03:31.971043 26920 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.006s
I20260927 23:03:31.976032 26935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:31.977856 26920 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260927 23:03:31.978142 26920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/wal
uuid: "b6bc59fe31e2493da1adca15be811cc1"
format_stamp: "Formatted at 2026-09-27 23:03:31 on dist-test-slave-5pd3"
server_key: "d53c10f4614e37bdc2e56fc875ec96fc"
server_key_iv: "545064d1181380c4411555fdeebeb4d0"
server_key_version: "encryptionkey@0"
I20260927 23:03:31.978441 26920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:31.992835 26920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:31.997205 26920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:31.997483 26920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:32.020395 26920 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.189:43853
I20260927 23:03:32.020457 26986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.189:43853 every 8 connection(s)
I20260927 23:03:32.021848 26920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
I20260927 23:03:32.026726 26987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:32.029270 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 26920
I20260927 23:03:32.029489 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-1/wal/instance
I20260927 23:03:32.029994 4394 external_mini_cluster.cc:1428] Setting key ff163ade4b641d97e8cf45e25fc6bcd6
I20260927 23:03:32.034814 26987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550211, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.74.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.188@KRBTEST.COM" created.
I20260927 23:03:32.063643 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.038179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34600 (local address 127.4.74.190:37757)
0927 23:03:32.038768 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.038772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.039926 (+ 1154us) server_negotiation.cc:407] Connection header received
0927 23:03:32.043317 (+ 3391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.043323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.043427 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.043648 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.046017 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.046920 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.049539 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.050062 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.050207 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.055476 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.055483 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.055487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.055647 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.059204 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.060306 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.060312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.060316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.060513 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.061044 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.061051 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.061057 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.061608 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.061788 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.062723 (+ 935us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.063041 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":329,"threads_started":1}
I20260927 23:03:32.064524 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.038618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37757 (local address 127.0.0.1:34600)
0927 23:03:32.039666 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.039734 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:32.042910 (+ 3176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:32.043943 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:32.043960 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:32.044809 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:32.045701 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.045728 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.047135 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.047144 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.049315 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.049333 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.050350 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.050360 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.050704 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.051642 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:32.051678 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:32.055074 (+ 3396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:32.059387 (+ 4313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.059394 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.059408 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.060129 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.060676 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.060679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.060681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.060905 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.061757 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:32.061767 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:32.062256 (+ 489us) client_negotiation.cc:769] Sending connection context
0927 23:03:32.062542 (+ 286us) client_negotiation.cc:240] Negotiation successful
0927 23:03:32.063073 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":365,"threads_started":1}
I20260927 23:03:32.066674 26987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550211, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
I20260927 23:03:32.084095 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.067758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:40066)
0927 23:03:32.067987 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.068022 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:32.068107 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:32.069248 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:32.069255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:32.069875 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:32.070150 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.070165 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.071742 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.071750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.072597 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.072608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.073647 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.073656 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.073904 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.074627 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:32.074655 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:32.077545 (+ 2890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:32.081169 (+ 3624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.081175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.081180 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.081554 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.081998 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.082008 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.082012 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.082166 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.082911 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:32.082916 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:32.083080 (+ 164us) client_negotiation.cc:769] Sending connection context
0927 23:03:32.083231 (+ 151us) client_negotiation.cc:240] Negotiation successful
0927 23:03:32.083596 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260927 23:03:32.084143 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.068062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40066 (local address 127.4.74.189:43853)
0927 23:03:32.068768 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.068774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.068798 (+ 24us) server_negotiation.cc:407] Connection header received
0927 23:03:32.068866 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.068872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.069039 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.069161 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.070369 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.071518 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.072776 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.073470 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.073595 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.077808 (+ 4213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.077821 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.077839 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.077871 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.081002 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.081695 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.081705 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.081709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.081875 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.082270 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.082277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.082281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.082765 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.082991 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.083400 (+ 409us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.083684 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":414,"threads_started":1}
I20260927 23:03:32.088100 26987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:03:32.089474 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.088916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58456)
0927 23:03:32.089157 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.089295 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260927 23:03:32.089757 26937 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
I20260927 23:03:32.091672 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.091232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58458)
0927 23:03:32.091396 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.091476 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 23:03:32.091931 26987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:45325: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab.
Entry for principal kudu/127.4.74.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab.
I20260927 23:03:32.100510 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.188:45325
--webserver_interface=127.4.74.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab
--principal=kudu/127.4.74.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:37757,127.4.74.189:43853,127.4.74.188:45325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:03:32.141331 26987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } attempt: 1
I20260927 23:03:32.142781 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.142221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58464)
0927 23:03:32.142446 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.142584 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260927 23:03:32.144116 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.143718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58468)
0927 23:03:32.143887 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.143967 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 23:03:32.144408 26987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:45325: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
I20260927 23:03:32.190217 26913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } attempt: 5
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550211, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
I20260927 23:03:32.212709 26987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } attempt: 2
I20260927 23:03:32.213632 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.191332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:40078)
0927 23:03:32.191657 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.191687 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:32.191890 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:32.192350 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:32.192357 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:32.192899 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:32.193167 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.193179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.194622 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.194628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.195697 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.195712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.196554 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.196561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.196811 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.202542 (+ 5731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:32.202570 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:32.205633 (+ 3063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:32.209827 (+ 4194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.209835 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.209839 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.210499 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.211034 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.211039 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.211043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.211238 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.212202 (+ 964us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:32.212208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:32.212426 (+ 218us) client_negotiation.cc:769] Sending connection context
0927 23:03:32.212686 (+ 260us) client_negotiation.cc:240] Negotiation successful
0927 23:03:32.213099 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260927 23:03:32.213948 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.213403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58472)
0927 23:03:32.213610 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.213713 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260927 23:03:32.214362 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.191502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.4.74.189:43853)
0927 23:03:32.191771 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.191780 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.191807 (+ 27us) server_negotiation.cc:407] Connection header received
0927 23:03:32.192080 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.192086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.192184 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.192317 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.193360 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.194418 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.195887 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.196368 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.196517 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.206191 (+ 9674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.206199 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.206203 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.206255 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.209576 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.210701 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.210707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.210709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.210878 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.211403 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.211407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.211410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.212048 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.212204 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.213788 (+ 1584us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.214079 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260927 23:03:32.215348 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.214957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58488)
0927 23:03:32.215115 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.215186 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260927 23:03:32.215673 26987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:45325: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
I20260927 23:03:32.215775 26913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:03:32.217208 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.216650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58498)
0927 23:03:32.216918 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.217010 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260927 23:03:32.218396 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.218016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58512)
0927 23:03:32.218193 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.218289 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260927 23:03:32.218703 26913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:45325: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
I20260927 23:03:32.270066 26913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } attempt: 1
I20260927 23:03:32.271634 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.270993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58516)
0927 23:03:32.271238 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.271349 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260927 23:03:32.272981 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.272611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58524)
0927 23:03:32.272785 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.272862 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260927 23:03:32.273257 26913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:45325: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
I20260927 23:03:32.307021 26987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } attempt: 3
I20260927 23:03:32.309000 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.308239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58536)
0927 23:03:32.308556 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.308721 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260927 23:03:32.310817 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.310196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58548)
0927 23:03:32.310497 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.310623 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
W20260927 23:03:32.311282 26987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:45325: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
I20260927 23:03:32.341547 26913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } attempt: 2
I20260927 23:03:32.343513 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.342871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58554)
0927 23:03:32.343167 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.343281 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260927 23:03:32.345312 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.344765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58570)
0927 23:03:32.345040 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.345138 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260927 23:03:32.345723 26913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:45325: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
I20260927 23:03:32.452136 26913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } attempt: 3
I20260927 23:03:32.453861 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.453108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58578)
0927 23:03:32.453491 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.453620 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
I20260927 23:03:32.454567 26987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } attempt: 4
I20260927 23:03:32.455654 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.454950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58586)
0927 23:03:32.455312 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.455406 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221}
W20260927 23:03:32.455917 26913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:45325: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
I20260927 23:03:32.456002 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.455363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58598)
0927 23:03:32.455679 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.455790 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260927 23:03:32.457028 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.456698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58614)
0927 23:03:32.456846 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.456920 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260927 23:03:32.457302 26987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:45325: Network error: Client connection negotiation failed: client connection to 127.4.74.188:45325: connect: Connection refused (error 111)
W20260927 23:03:32.462153 26995 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 23:03:32.462538 26995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:32.462674 26995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:32.462812 26995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:32.470656 26995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:32.470774 26995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:32.470868 26995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:32.470948 26995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:32.471069 26995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:32.481782 26995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:37757,127.4.74.189:43853,127.4.74.188:45325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab
--principal=kudu/127.4.74.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.188:45325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.74.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:32.483207 26995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:32.484612 26995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:32.497730 27000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:32.498034 27001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:32.498257 27003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:32.499161 26995 server_base.cc:1034] running on GCE node
I20260927 23:03:32.500442 26995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:32.502462 26995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:32.503754 26995 hybrid_clock.cc:648] HybridClock initialized: now 1790550212503671 us; error 52 us; skew 500 ppm
I20260927 23:03:32.504205 26995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550212, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:32.508214 26995 init.cc:377] Logged in from keytab as kudu/127.4.74.188@KRBTEST.COM (short username kudu)
I20260927 23:03:32.510141 26995 webserver.cc:460] Webserver started at http://127.4.74.188:42483/ using document root <none> and password file <none>
I20260927 23:03:32.511052 26995 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:32.511206 26995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:32.511646 26995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:32.515235 26995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/data/instance:
uuid: "a31fcf35614a44f082bc66cc563adad6"
format_stamp: "Formatted at 2026-09-27 23:03:32 on dist-test-slave-5pd3"
server_key: "feaeea5c719c2e56a01b00e5633e2737"
server_key_iv: "221a384dfe8b074529b42ae0ed8e4d0e"
server_key_version: "encryptionkey@0"
I20260927 23:03:32.516260 26995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/wal/instance:
uuid: "a31fcf35614a44f082bc66cc563adad6"
format_stamp: "Formatted at 2026-09-27 23:03:32 on dist-test-slave-5pd3"
server_key: "feaeea5c719c2e56a01b00e5633e2737"
server_key_iv: "221a384dfe8b074529b42ae0ed8e4d0e"
server_key_version: "encryptionkey@0"
I20260927 23:03:32.522857 26995 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260927 23:03:32.527808 27010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:32.529681 26995 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:03:32.529961 26995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/wal
uuid: "a31fcf35614a44f082bc66cc563adad6"
format_stamp: "Formatted at 2026-09-27 23:03:32 on dist-test-slave-5pd3"
server_key: "feaeea5c719c2e56a01b00e5633e2737"
server_key_iv: "221a384dfe8b074529b42ae0ed8e4d0e"
server_key_version: "encryptionkey@0"
I20260927 23:03:32.530274 26995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:32.557906 26995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:32.563109 26995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:32.563479 26995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:32.586297 27061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.188:45325 every 8 connection(s)
I20260927 23:03:32.586337 26995 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.188:45325
I20260927 23:03:32.588230 26995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
I20260927 23:03:32.592658 27062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:32.598527 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 26995
I20260927 23:03:32.598790 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790549821755302-4394-0/minicluster-data/master-2/wal/instance
I20260927 23:03:32.599310 4394 external_mini_cluster.cc:1428] Setting key d484c0765bb6047c8a312acf49140d1d
I20260927 23:03:32.599756 27062 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550212, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
I20260927 23:03:32.623863 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.602892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.4.74.190:37757)
0927 23:03:32.604563 (+ 1671us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.604568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.604590 (+ 22us) server_negotiation.cc:407] Connection header received
0927 23:03:32.605533 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.605537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.605612 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.605777 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.607468 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.608287 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.610138 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.610738 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.610839 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.615452 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.615458 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.615462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.615508 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.619254 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.620637 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.620642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.620645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.620869 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.621489 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.621493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.621495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.621977 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.622171 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.623030 (+ 859us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.623357 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1566,"thread_start_us":1414,"threads_started":1}
I20260927 23:03:32.625193 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.601516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58624 (local address 127.4.74.188:45325)
0927 23:03:32.602462 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.602477 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.602571 (+ 94us) server_negotiation.cc:407] Connection header received
0927 23:03:32.605189 (+ 2618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.605227 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.605607 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.605847 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.607369 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.609029 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.610573 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.611381 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.611521 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.616120 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.616134 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.616157 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.616209 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.620136 (+ 3927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.620949 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.620963 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.620970 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.621199 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.621654 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.621659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.621663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.622547 (+ 884us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.622824 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.623292 (+ 468us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.623780 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":183,"server-negotiator.queue_time_us":696,"spinlock_wait_cycles":24448,"thread_start_us":433,"threads_started":1}
I20260927 23:03:32.625217 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.602741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:37757 (local address 127.0.0.1:34612)
0927 23:03:32.603693 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.603763 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:32.605189 (+ 1426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:32.605838 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:32.605846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:32.606797 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:32.607241 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.607262 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.608474 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.608481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.609966 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.609978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.610993 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.611000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.611286 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.612319 (+ 1033us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:32.612366 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:32.615162 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:32.619489 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.619507 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.619519 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.620386 (+ 867us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.621017 (+ 631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.621020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.621023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.621310 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.622176 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:32.622184 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:32.622493 (+ 309us) client_negotiation.cc:769] Sending connection context
0927 23:03:32.622898 (+ 405us) client_negotiation.cc:240] Negotiation successful
0927 23:03:32.623597 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"spinlock_wait_cycles":19840,"thread_start_us":347,"threads_started":1}
I20260927 23:03:32.627938 27062 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:03:32.633173 26913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } attempt: 4
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550212, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550211, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
I20260927 23:03:32.647310 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.629010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40086 (local address 127.4.74.189:43853)
0927 23:03:32.629289 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.629303 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.629357 (+ 54us) server_negotiation.cc:407] Connection header received
0927 23:03:32.629624 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.629630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.629733 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.629798 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.630782 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.631619 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.633111 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.633758 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.633928 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.639111 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.639122 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.639127 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.639180 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.643062 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.644290 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.644298 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.644304 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.644513 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.645048 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.645055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.645060 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.645529 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.645719 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.646508 (+ 789us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.646839 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260927 23:03:32.647495 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.629659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.4.74.190:37757)
0927 23:03:32.630164 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.630169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.630183 (+ 14us) server_negotiation.cc:407] Connection header received
0927 23:03:32.630249 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.630253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.630327 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.630409 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.631780 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.632503 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.634176 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.634865 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.635004 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.640984 (+ 5980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.640993 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.641000 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.641052 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.644424 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.645376 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.645384 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.645388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.645523 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.645896 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.645901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.645905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.646223 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.646533 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.646852 (+ 319us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.647108 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396}
I20260927 23:03:32.648412 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.629026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:43853 (local address 127.0.0.1:40086)
0927 23:03:32.629241 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.629266 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:32.629436 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:32.629971 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:32.629974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:32.630361 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:32.630594 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.630607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.631809 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.631813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.632920 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.632935 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.634219 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.634225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.634477 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.635305 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:32.635328 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:32.638804 (+ 3476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:32.643233 (+ 4429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.643245 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.643249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.644044 (+ 795us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.644650 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.644655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.644661 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.644893 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.645804 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:32.645813 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:32.646013 (+ 200us) client_negotiation.cc:769] Sending connection context
0927 23:03:32.647594 (+ 1581us) client_negotiation.cc:240] Negotiation successful
0927 23:03:32.647990 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260927 23:03:32.650085 27062 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:03:32.650914 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.633944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58632)
0927 23:03:32.634237 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.634255 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:32.634385 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:32.635172 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:32.635177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:32.635734 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:32.636015 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.636029 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.637126 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.637132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.638108 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.638120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.638849 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.638855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.639038 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.639667 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:32.639696 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:32.642770 (+ 3074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:32.647122 (+ 4352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.647129 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.647133 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.647801 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.648308 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.648315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.648318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.648938 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.649773 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:32.649779 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:32.649969 (+ 190us) client_negotiation.cc:769] Sending connection context
0927 23:03:32.650095 (+ 126us) client_negotiation.cc:240] Negotiation successful
0927 23:03:32.650498 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260927 23:03:32.652176 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.634069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.4.74.188:45325)
0927 23:03:32.634267 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.634273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.634327 (+ 54us) server_negotiation.cc:407] Connection header received
0927 23:03:32.634506 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.634511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.634591 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.634716 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.636219 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.636968 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.638257 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.638694 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.638832 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.643092 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.643099 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.643107 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.643160 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.646918 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.647982 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.647987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.647990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.648149 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.649111 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.649117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.649120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.649598 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.651310 (+ 1712us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.651444 (+ 134us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.651872 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260927 23:03:32.659852 26913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2: Bootstrap starting.
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550212, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
I20260927 23:03:32.663666 26913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:32.665052 26913 log.cc:824] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:32.668018 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.650924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58634)
0927 23:03:32.651428 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.651451 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:32.651608 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:32.652503 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:32.652511 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:32.653008 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:32.653210 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.653222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.655229 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.655236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.656674 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.656692 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.657565 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.657570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.657792 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.658390 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:32.658409 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:32.661188 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:32.665016 (+ 3828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.665024 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.665031 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.665657 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.666124 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.666132 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.666135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.666274 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.666915 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:32.666919 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:32.667066 (+ 147us) client_negotiation.cc:769] Sending connection context
0927 23:03:32.667205 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 23:03:32.667523 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375}
I20260927 23:03:32.668675 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.651266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58634 (local address 127.4.74.188:45325)
0927 23:03:32.652204 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.652210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.652232 (+ 22us) server_negotiation.cc:407] Connection header received
0927 23:03:32.652275 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.652280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.652352 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.653347 (+ 995us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.653611 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.655056 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.656852 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.657392 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.657540 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.661440 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.661447 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.661453 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.661497 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.664820 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.665877 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.665881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.665884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.666005 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.666386 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.666393 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.666400 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.666779 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.667783 (+ 1004us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.667936 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.668239 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":595,"threads_started":1}
I20260927 23:03:32.668820 26913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2: No bootstrap required, opened a new log
I20260927 23:03:32.670989 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.649564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40088 (local address 127.4.74.189:43853)
0927 23:03:32.649751 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.649756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.649777 (+ 21us) server_negotiation.cc:407] Connection header received
0927 23:03:32.649955 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.649960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.650039 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.650158 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.651860 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.653068 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.655461 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.656029 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.656178 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.662943 (+ 6765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.662951 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.662956 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.663010 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.666841 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.667719 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.667725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.667728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.667910 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.668859 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.668867 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.668873 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.669307 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.669624 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.670312 (+ 688us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.670596 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260927 23:03:32.673916 27062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6: Bootstrap starting.
I20260927 23:03:32.678367 27062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:32.678180 26913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } }
I20260927 23:03:32.678653 26913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34f227b94537444cb1c0f569b3a3c2d2, State: Initialized, Role: FOLLOWER
I20260927 23:03:32.679227 26913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } }
I20260927 23:03:32.680001 27062 log.cc:824] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:32.680601 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:32.680507 27070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } } }
I20260927 23:03:32.681358 27070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:32.681743 26913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:32.684916 26888 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "34f227b94537444cb1c0f569b3a3c2d2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34620
I20260927 23:03:32.685355 26888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 23:03:32.685596 26888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:32.688879 26888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } }
I20260927 23:03:32.691668 26888 leader_election.cc:288] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [CANDIDATE]: Term 1 election: Requested vote from peers b6bc59fe31e2493da1adca15be811cc1 (127.4.74.189:43853), a31fcf35614a44f082bc66cc563adad6 (127.4.74.188:45325)
I20260927 23:03:32.692914 27062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6: No bootstrap required, opened a new log
W20260927 23:03:32.698534 26867 tablet.cc:2369] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:03:32.710354 27062 raft_consensus.cc:348] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } }
I20260927 23:03:32.711004 27062 raft_consensus.cc:729] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31fcf35614a44f082bc66cc563adad6, State: Initialized, Role: FOLLOWER
I20260927 23:03:32.711670 27062 consensus_queue.cc:260] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } }
W20260927 23:03:32.712092 27082 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:03:32.712288 27082 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:03:32.713482 27086 sys_catalog.cc:455] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } } }
I20260927 23:03:32.714061 27086 sys_catalog.cc:458] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:32.715018 27062 sys_catalog.cc:564] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:32.715126 27037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34f227b94537444cb1c0f569b3a3c2d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a31fcf35614a44f082bc66cc563adad6"
I20260927 23:03:32.715888 27037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:32.722560 27037 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f227b94537444cb1c0f569b3a3c2d2 in term 1.
I20260927 23:03:32.724062 26863 leader_election.cc:302] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f227b94537444cb1c0f569b3a3c2d2, a31fcf35614a44f082bc66cc563adad6; no voters:
I20260927 23:03:32.724831 27070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:32.729676 27070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [term 1 LEADER]: Becoming Leader. State: Replica: 34f227b94537444cb1c0f569b3a3c2d2, State: Running, Role: LEADER
I20260927 23:03:32.730685 27070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } }
I20260927 23:03:32.744889 27089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34f227b94537444cb1c0f569b3a3c2d2. Latest consensus state: current_term: 1 leader_uuid: "34f227b94537444cb1c0f569b3a3c2d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } } }
I20260927 23:03:32.745683 27089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:32.697252 26846 master_runner.cc:186] Error getting master registration for 127.4.74.189:43853: OK, instance_id { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" instance_seqno: 1790550212013869 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 23:03:32.750152 26846 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 23:03:32.756522 26987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } attempt: 5
Sep 27 23:03:32 dist-test-slave-5pd3 krb5kdc[26833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550211, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
I20260927 23:03:32.784742 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.758575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.4.74.188:45325)
0927 23:03:32.758811 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.758820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.758847 (+ 27us) server_negotiation.cc:407] Connection header received
0927 23:03:32.758947 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.758955 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.759087 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.759223 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.761161 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.762806 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.765584 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.766677 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.766846 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.773706 (+ 6860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.773719 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.773727 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.773800 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.779499 (+ 5699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.780931 (+ 1432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.780941 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.780948 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.781143 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.781700 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.781708 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.781713 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.782386 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.782593 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.783765 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.784136 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260927 23:03:32.786015 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.757520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:45325 (local address 127.0.0.1:58638)
0927 23:03:32.757934 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:32.757962 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:32.758154 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:32.759494 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:32.759503 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:32.760506 (+ 1003us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:32.760853 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.760872 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.763160 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.763168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.765310 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:32.765334 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.767232 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.767242 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:32.767621 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.768682 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:32.768719 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:32.773291 (+ 4572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:32.779833 (+ 6542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.779844 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.779852 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.780721 (+ 869us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.781323 (+ 602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:32.781331 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:32.781338 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:32.781540 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:32.782666 (+ 1126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:32.782675 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:32.783049 (+ 374us) client_negotiation.cc:769] Sending connection context
0927 23:03:32.783266 (+ 217us) client_negotiation.cc:240] Negotiation successful
0927 23:03:32.785437 (+ 2171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234}
I20260927 23:03:32.786732 27093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:32.789214 27093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:32.795429 26987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1: Bootstrap starting.
I20260927 23:03:32.801944 26987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:32.813932 26987 log.cc:824] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1: Log is configured to *not* fsync() on all Append() calls
W20260927 23:03:32.816097 27016 tablet.cc:2369] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:03:32.833885 26987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1: No bootstrap required, opened a new log
I20260927 23:03:32.834549 27037 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [term 1 FOLLOWER]: Refusing update from remote peer 34f227b94537444cb1c0f569b3a3c2d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:03:32.837123 27089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:03:32.838641 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.761750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40092 (local address 127.4.74.189:43853)
0927 23:03:32.762024 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.762034 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.779253 (+ 17219us) server_negotiation.cc:407] Connection header received
0927 23:03:32.780359 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.780364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.780472 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.780556 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.794684 (+ 14128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.795911 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.815077 (+ 19166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.815931 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.816087 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.823255 (+ 7168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.823265 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.823273 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.823350 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.828576 (+ 5226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.830815 (+ 2239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.830826 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.830831 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.833589 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.834574 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.834602 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.834607 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.835523 (+ 916us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.835983 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.837274 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.837792 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260927 23:03:32.846339 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.753842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.4.74.188:45325)
0927 23:03:32.754175 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.754185 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.774870 (+ 20685us) server_negotiation.cc:407] Connection header received
0927 23:03:32.776147 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.776157 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.776334 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.776449 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.790283 (+ 13834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.791411 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.809362 (+ 17951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.810372 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.810514 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.832447 (+ 21933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.832456 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.832464 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.832516 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.842425 (+ 9909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.843647 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.843653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.843659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.843834 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.844368 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.844376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.844381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.844783 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.845034 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.845762 (+ 728us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.846010 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260927 23:03:32.852013 26987 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } }
I20260927 23:03:32.854245 26987 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6bc59fe31e2493da1adca15be811cc1, State: Initialized, Role: FOLLOWER
I20260927 23:03:32.855301 26987 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } }
I20260927 23:03:32.858238 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:32.753314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.4.74.190:37757)
0927 23:03:32.753547 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:32.753557 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:32.774697 (+ 21140us) server_negotiation.cc:407] Connection header received
0927 23:03:32.776784 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:32.776793 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:32.776909 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:32.777000 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:32.788720 (+ 11720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.789939 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.807467 (+ 17528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:32.808536 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:32.808722 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:32.832078 (+ 23356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:32.832087 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:32.832095 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:32.832145 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:32.844825 (+ 12680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.850078 (+ 5253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.850128 (+ 50us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.850135 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.850707 (+ 572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:32.852394 (+ 1687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:32.852404 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:32.852411 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:32.854049 (+ 1638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:32.854982 (+ 933us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:32.856650 (+ 1668us) server_negotiation.cc:299] Negotiation successful
0927 23:03:32.857252 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 23:03:32.872822 27107 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } } }
I20260927 23:03:32.873423 27107 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:32.874804 26987 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:32.875293 27086 sys_catalog.cc:455] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34f227b94537444cb1c0f569b3a3c2d2. Latest consensus state: current_term: 1 leader_uuid: "34f227b94537444cb1c0f569b3a3c2d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } } }
I20260927 23:03:32.875813 27086 sys_catalog.cc:458] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:32.879817 26962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34f227b94537444cb1c0f569b3a3c2d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b6bc59fe31e2493da1adca15be811cc1"
I20260927 23:03:32.880566 26962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:32.883399 27099 mvcc.cc:205] Tried to move back new op lower bound from 7334093671746502656 to 7334093671359033344. Current Snapshot: MvccSnapshot[applied={T|T < 7334093671746502656}]
I20260927 23:03:32.886380 27105 mvcc.cc:205] Tried to move back new op lower bound from 7334093671746502656 to 7334093671359033344. Current Snapshot: MvccSnapshot[applied={T|T < 7334093671746502656}]
I20260927 23:03:32.888374 26962 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f227b94537444cb1c0f569b3a3c2d2 in term 1.
I20260927 23:03:32.889269 26961 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [term 1 FOLLOWER]: Refusing update from remote peer 34f227b94537444cb1c0f569b3a3c2d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:03:32.889806 27086 sys_catalog.cc:455] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34f227b94537444cb1c0f569b3a3c2d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } } }
I20260927 23:03:32.890180 27086 sys_catalog.cc:458] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:32.893481 27070 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:03:32.894936 27093 catalog_manager.cc:1347] Generated new cluster ID: 577c60211cce40649f16de164af9c31a
I20260927 23:03:32.895089 27093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260927 23:03:32.902716 27110 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:03:32.903153 27110 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:03:32.916929 27089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34f227b94537444cb1c0f569b3a3c2d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } } }
I20260927 23:03:32.917426 27089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:32.927103 27093 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:32.929673 27093 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:32.927084 26995 master_runner.cc:186] Error getting master registration for 127.4.74.189:43853: OK, instance_id { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" instance_seqno: 1790550212013869 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 23:03:32.940620 26995 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 23:03:32.957579 27093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2: Generated new TSK 0
I20260927 23:03:32.959435 27093 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:03:32.974129 27107 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34f227b94537444cb1c0f569b3a3c2d2. Latest consensus state: current_term: 1 leader_uuid: "34f227b94537444cb1c0f569b3a3c2d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } } }
I20260927 23:03:32.974593 27107 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:32.974845 27089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34f227b94537444cb1c0f569b3a3c2d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } } }
I20260927 23:03:32.975234 27089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f227b94537444cb1c0f569b3a3c2d2 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:32.975010 27123 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34f227b94537444cb1c0f569b3a3c2d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34f227b94537444cb1c0f569b3a3c2d2" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 37757 } } peers { permanent_uuid: "b6bc59fe31e2493da1adca15be811cc1" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 43853 } } peers { permanent_uuid: "a31fcf35614a44f082bc66cc563adad6" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 45325 } } }
I20260927 23:03:32.975351 27123 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:32.991339 27129 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260927 23:03:33.080183 27068 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.062569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58642 (local address 127.4.74.188:45325)
0927 23:03:33.062746 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.062752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.077913 (+ 15161us) server_negotiation.cc:407] Connection header received
0927 23:03:33.078437 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.078443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.078524 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.078598 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.079857 (+ 1259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58642: BlockingRecv error: recv got EOF from 127.0.0.1:58642 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260927 23:03:33.081892 27067 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.062247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34638 (local address 127.4.74.190:37757)
0927 23:03:33.062461 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.062468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.077232 (+ 14764us) server_negotiation.cc:407] Connection header received
0927 23:03:33.077454 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.077462 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.077566 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.077678 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.081639 (+ 3961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34638: BlockingRecv error: recv got EOF from 127.0.0.1:34638 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260927 23:03:33.084668 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.069287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.4.74.189:43853)
0927 23:03:33.069495 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.069501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.082750 (+ 13249us) server_negotiation.cc:407] Connection header received
0927 23:03:33.083277 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.083283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.083383 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.083460 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.084442 (+ 982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40104: BlockingRecv error: recv got EOF from 127.0.0.1:40104 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260927 23:03:33.098039 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.091705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40110 (local address 127.4.74.189:43853)
0927 23:03:33.091904 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.091910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.096224 (+ 4314us) server_negotiation.cc:407] Connection header received
0927 23:03:33.096422 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.096429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.096615 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.096784 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.097693 (+ 909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40110: BlockingRecv error: recv got EOF from 127.0.0.1:40110 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260927 23:03:33.114255 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.106190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.4.74.189:43853)
0927 23:03:33.106449 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.106457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.112390 (+ 5933us) server_negotiation.cc:407] Connection header received
0927 23:03:33.112643 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.112649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.112763 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.112913 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.113988 (+ 1075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40120: BlockingRecv error: recv got EOF from 127.0.0.1:40120 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260927 23:03:33.126102 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.122036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40136 (local address 127.4.74.189:43853)
0927 23:03:33.122276 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.122284 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.124282 (+ 1998us) server_negotiation.cc:407] Connection header received
0927 23:03:33.124507 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.124516 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.124644 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.124788 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.125832 (+ 1044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40136: BlockingRecv error: recv got EOF from 127.0.0.1:40136 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260927 23:03:33.142189 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.134074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40140 (local address 127.4.74.189:43853)
0927 23:03:33.134286 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.134292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.140415 (+ 6123us) server_negotiation.cc:407] Connection header received
0927 23:03:33.140657 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.140665 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.140780 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.140922 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.141934 (+ 1012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40140: BlockingRecv error: recv got EOF from 127.0.0.1:40140 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260927 23:03:33.157155 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.154393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.4.74.189:43853)
0927 23:03:33.154636 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.154643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.155371 (+ 728us) server_negotiation.cc:407] Connection header received
0927 23:03:33.155652 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.155659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.155780 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.155910 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.156914 (+ 1004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40148: BlockingRecv error: recv got EOF from 127.0.0.1:40148 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260927 23:03:33.184653 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.181857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.4.74.189:43853)
0927 23:03:33.182117 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.182126 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.182814 (+ 688us) server_negotiation.cc:407] Connection header received
0927 23:03:33.183069 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.183075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.183185 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.183351 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.184410 (+ 1059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40156: BlockingRecv error: recv got EOF from 127.0.0.1:40156 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260927 23:03:33.225049 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.221985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40166 (local address 127.4.74.189:43853)
0927 23:03:33.222292 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.222299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.223075 (+ 776us) server_negotiation.cc:407] Connection header received
0927 23:03:33.223426 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.223433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.223584 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.223737 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.224754 (+ 1017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40166: BlockingRecv error: recv got EOF from 127.0.0.1:40166 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
W20260927 23:03:33.300611 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.297630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.4.74.189:43853)
0927 23:03:33.297889 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.297897 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.298650 (+ 753us) server_negotiation.cc:407] Connection header received
0927 23:03:33.298915 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.298922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.299041 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.299187 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.300369 (+ 1182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40170: BlockingRecv error: recv got EOF from 127.0.0.1:40170 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260927 23:03:33.435477 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.432935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40186 (local address 127.4.74.189:43853)
0927 23:03:33.433135 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.433139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.433533 (+ 394us) server_negotiation.cc:407] Connection header received
0927 23:03:33.433757 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.433762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.433840 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.433964 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.435088 (+ 1124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40186: BlockingRecv error: recv got EOF from 127.0.0.1:40186 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260927 23:03:33.698462 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:33.696457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.4.74.189:43853)
0927 23:03:33.696750 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:33.696756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:33.696900 (+ 144us) server_negotiation.cc:407] Connection header received
0927 23:03:33.697089 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:33.697094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:33.697174 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:33.697286 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:33.698224 (+ 938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40200: BlockingRecv error: recv got EOF from 127.0.0.1:40200 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
I20260927 23:03:33.905473 27110 catalog_manager.cc:1259] Loaded cluster ID: 577c60211cce40649f16de164af9c31a
I20260927 23:03:33.905609 27110 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6: loading cluster ID for follower catalog manager: success
I20260927 23:03:33.908916 27110 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6: acquiring CA information for follower catalog manager: success
I20260927 23:03:33.911311 27110 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a31fcf35614a44f082bc66cc563adad6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:03:33.996985 27129 catalog_manager.cc:1259] Loaded cluster ID: 577c60211cce40649f16de164af9c31a
I20260927 23:03:33.997155 27129 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1: loading cluster ID for follower catalog manager: success
I20260927 23:03:34.000583 27129 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1: acquiring CA information for follower catalog manager: success
I20260927 23:03:34.002964 27129 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b6bc59fe31e2493da1adca15be811cc1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:03:34.225533 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:34.217769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40206 (local address 127.4.74.189:43853)
0927 23:03:34.218765 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:34.218770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:34.218789 (+ 19us) server_negotiation.cc:407] Connection header received
0927 23:03:34.218869 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:34.218873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:34.219072 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:34.219244 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:03:34.219762 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:34.220944 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:34.222606 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:34.223182 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:34.223320 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:34.223953 (+ 633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:03:34.224697 (+ 744us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:03:34.224872 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:34.224930 (+ 58us) server_negotiation.cc:299] Negotiation successful
0927 23:03:34.225040 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":570,"threads_started":1}
I20260927 23:03:34.251263 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:34.240430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58656 (local address 127.4.74.188:45325)
0927 23:03:34.244037 (+ 3607us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:34.244045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:34.244069 (+ 24us) server_negotiation.cc:407] Connection header received
0927 23:03:34.244204 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:34.244210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:34.244463 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:34.244689 (+ 226us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:03:34.245347 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:34.246519 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:34.248214 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:34.248859 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:34.249021 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:34.249611 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:03:34.250407 (+ 796us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:03:34.250576 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:34.250810 (+ 234us) server_negotiation.cc:299] Negotiation successful
0927 23:03:34.250909 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3447,"thread_start_us":3302,"threads_started":1}
I20260927 23:03:34.277084 27210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:34.260043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34644 (local address 127.4.74.190:37757)
0927 23:03:34.267630 (+ 7587us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:34.267641 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:34.267672 (+ 31us) server_negotiation.cc:407] Connection header received
0927 23:03:34.267863 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:34.267871 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:34.268212 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:34.268503 (+ 291us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:03:34.270076 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:34.271242 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:34.273618 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:34.274290 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:34.274420 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:34.275299 (+ 879us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:03:34.275983 (+ 684us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:03:34.276235 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:34.276548 (+ 313us) server_negotiation.cc:299] Negotiation successful
0927 23:03:34.276688 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7373,"thread_start_us":7252,"threads_started":1}
I20260927 23:03:34.277966 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:34.260164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58666 (local address 127.4.74.188:45325)
0927 23:03:34.264868 (+ 4704us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:34.264876 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:34.270352 (+ 5476us) server_negotiation.cc:407] Connection header received
0927 23:03:34.270452 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:34.270458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:34.270593 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:34.270685 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:03:34.271587 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:34.272716 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:34.275087 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:34.275772 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:34.275877 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:34.276795 (+ 918us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:03:34.276948 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:03:34.277037 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:34.277542 (+ 505us) server_negotiation.cc:299] Negotiation successful
0927 23:03:34.277645 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4521}
I20260927 23:03:34.281771 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:34.259773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.4.74.189:43853)
0927 23:03:34.259985 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:34.259992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:34.272456 (+ 12464us) server_negotiation.cc:407] Connection header received
0927 23:03:34.273093 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:34.273100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:34.273210 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:34.273304 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:03:34.274282 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:34.275651 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:34.278975 (+ 3324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:34.279767 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:34.279917 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:34.280711 (+ 794us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:03:34.280901 (+ 190us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:03:34.281033 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:34.281236 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 23:03:34.281344 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 23:03:34.283985 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26846
I20260927 23:03:34.285468 26908 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:03:34.417654 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 26846
W20260927 23:03:34.440027 27014 connection.cc:535] server connection from 127.0.0.1:58632 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 23:03:34.440479 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26920
I20260927 23:03:34.441702 26982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:03:34.566689 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 26920
I20260927 23:03:34.587423 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 26995
I20260927 23:03:34.588698 27057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:03:34.699331 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 26995
2026-09-27T23:03:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5182 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 23:03:34 dist-test-slave-5pd3 krb5kdc[27216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 23:03:34 dist-test-slave-5pd3 krb5kdc[27216](info): set up 2 sockets
Sep 27 23:03:34 dist-test-slave-5pd3 krb5kdc[27216](info): commencing operation
krb5kdc: starting...
W20260927 23:03:35.844218 4394 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.006s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/test-user.keytab.
Sep 27 23:03:36 dist-test-slave-5pd3 krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T23:03:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:03:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
Entry for principal kudu/127.4.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab.
I20260927 23:03:36.185391 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36233
--webserver_interface=127.4.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:35105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36233,127.4.74.189:41733,127.4.74.188:42203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 23:03:36.603077 27229 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 23:03:36.603646 27229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:36.603749 27229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:36.603847 27229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:36.612772 27229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:36.612933 27229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:36.613004 27229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:36.613058 27229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:36.613123 27229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:36.625509 27229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36233,127.4.74.189:41733,127.4.74.188:42203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.190.keytab
--principal=kudu/127.4.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.190:36233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:36.627249 27229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:36.628945 27229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:36.644035 27234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:36.643919 27235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:36.644070 27237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:36.645542 27229 server_base.cc:1034] running on GCE node
I20260927 23:03:36.646571 27229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:36.648355 27229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:36.649830 27229 hybrid_clock.cc:648] HybridClock initialized: now 1790550216649668 us; error 62 us; skew 500 ppm
I20260927 23:03:36.650254 27229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:36 dist-test-slave-5pd3 krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550216, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:36.654131 27229 init.cc:377] Logged in from keytab as kudu/127.4.74.190@KRBTEST.COM (short username kudu)
I20260927 23:03:36.656098 27229 webserver.cc:460] Webserver started at http://127.4.74.190:35669/ using document root <none> and password file <none>
I20260927 23:03:36.656968 27229 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:36.657123 27229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:36.657567 27229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:36.661125 27229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/data/instance:
uuid: "d8956e81eccd4cf2925831ac07b1da17"
format_stamp: "Formatted at 2026-09-27 23:03:36 on dist-test-slave-5pd3"
server_key: "238a74a018066faa4962a6ac8068bdff"
server_key_iv: "6b4e5d31cee821a1696e355ab480b8d6"
server_key_version: "encryptionkey@0"
I20260927 23:03:36.662050 27229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/wal/instance:
uuid: "d8956e81eccd4cf2925831ac07b1da17"
format_stamp: "Formatted at 2026-09-27 23:03:36 on dist-test-slave-5pd3"
server_key: "238a74a018066faa4962a6ac8068bdff"
server_key_iv: "6b4e5d31cee821a1696e355ab480b8d6"
server_key_version: "encryptionkey@0"
I20260927 23:03:36.668347 27229 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260927 23:03:36.673691 27244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:36.675807 27229 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.003s
I20260927 23:03:36.676080 27229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/wal
uuid: "d8956e81eccd4cf2925831ac07b1da17"
format_stamp: "Formatted at 2026-09-27 23:03:36 on dist-test-slave-5pd3"
server_key: "238a74a018066faa4962a6ac8068bdff"
server_key_iv: "6b4e5d31cee821a1696e355ab480b8d6"
server_key_version: "encryptionkey@0"
I20260927 23:03:36.676370 27229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:36.689458 27229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:36.694322 27229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:36.694669 27229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:36.718395 27229 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.190:36233
I20260927 23:03:36.718442 27295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.190:36233 every 8 connection(s)
I20260927 23:03:36.720028 27229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/data/info.pb
I20260927 23:03:36.721825 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 27229
I20260927 23:03:36.722060 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-0/wal/instance
I20260927 23:03:36.722671 4394 external_mini_cluster.cc:1428] Setting key 09a05e8a322c458063488c86aa4297d5
I20260927 23:03:36.725862 27296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:36.737788 27296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:03:36 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550216, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.74.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.189@KRBTEST.COM" created.
I20260927 23:03:36.765985 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:36.742025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49766 (local address 127.4.74.190:36233)
0927 23:03:36.744210 (+ 2185us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:36.744220 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:36.744279 (+ 59us) server_negotiation.cc:407] Connection header received
0927 23:03:36.746119 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:36.746126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:36.746421 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:36.746647 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:36.748202 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:36.749520 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:36.751051 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:36.751778 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:36.752020 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:36.755984 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:36.756017 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:36.756041 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:36.756083 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:36.759960 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:36.761394 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:36.761402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:36.761404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:36.761628 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:36.762328 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:36.762336 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:36.762339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:36.763055 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:36.763273 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:36.764311 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0927 23:03:36.764685 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1901,"thread_start_us":1645,"threads_started":1}
I20260927 23:03:36.766120 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:36.742023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36233 (local address 127.0.0.1:49766)
0927 23:03:36.743135 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:36.743204 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:36.745651 (+ 2447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:36.746677 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:36.746684 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:36.747514 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:36.747968 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:36.747989 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:36.749705 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:36.749714 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:36.750900 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:36.750913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:36.752053 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:36.752060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:36.752260 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:36.753026 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:36.753052 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:36.755710 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:36.760220 (+ 4510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:36.760248 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:36.760270 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:36.761128 (+ 858us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:36.761783 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:36.761792 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:36.761800 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:36.762103 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:36.763265 (+ 1162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:36.763280 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:36.763697 (+ 417us) client_negotiation.cc:769] Sending connection context
0927 23:03:36.764075 (+ 378us) client_negotiation.cc:240] Negotiation successful
0927 23:03:36.764979 (+ 904us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":157,"thread_start_us":514,"threads_started":1}
I20260927 23:03:36.769770 27296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:03:36.771481 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:36.770744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42028)
0927 23:03:36.771046 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:36.771249 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260927 23:03:36.772034 27248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
I20260927 23:03:36.774134 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:36.773508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42038)
0927 23:03:36.773751 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:36.773886 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260927 23:03:36.774428 27296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:41733: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab.
Entry for principal kudu/127.4.74.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab.
I20260927 23:03:36.803682 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.189:41733
--webserver_interface=127.4.74.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab
--principal=kudu/127.4.74.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:35105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36233,127.4.74.189:41733,127.4.74.188:42203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:03:36.823931 27296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } attempt: 1
I20260927 23:03:36.825565 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:36.824719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42040)
0927 23:03:36.825085 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:36.825308 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260927 23:03:36.827104 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:36.826478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42048)
0927 23:03:36.826710 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:36.826832 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260927 23:03:36.827378 27296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:41733: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
I20260927 23:03:36.895718 27296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } attempt: 2
I20260927 23:03:36.897400 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:36.896629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42060)
0927 23:03:36.896975 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:36.897159 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260927 23:03:36.898976 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:36.898458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42072)
0927 23:03:36.898639 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:36.898764 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260927 23:03:36.899498 27296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:41733: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
I20260927 23:03:36.991065 27296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } attempt: 3
I20260927 23:03:36.992717 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:36.991879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42088)
0927 23:03:36.992303 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:36.992464 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260927 23:03:36.994197 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:36.993547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42096)
0927 23:03:36.993792 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:36.993961 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260927 23:03:36.994513 27296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:41733: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
I20260927 23:03:37.137807 27296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } attempt: 4
I20260927 23:03:37.139469 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.138666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42104)
0927 23:03:37.139117 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.139267 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240}
I20260927 23:03:37.140885 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.140402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42106)
0927 23:03:37.140604 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.140719 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260927 23:03:37.141203 27296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.189:41733: Network error: Client connection negotiation failed: client connection to 127.4.74.189:41733: connect: Connection refused (error 111)
W20260927 23:03:37.227645 27303 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 23:03:37.228078 27303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:37.228245 27303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:37.228387 27303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:37.236016 27303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:37.236124 27303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:37.236181 27303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:37.236294 27303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:37.236394 27303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:37.247229 27303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36233,127.4.74.189:41733,127.4.74.188:42203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.189.keytab
--principal=kudu/127.4.74.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.189:41733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.74.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:37.248814 27303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:37.250239 27303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:37.263732 27308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:37.263685 27309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:37.264366 27311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:37.265166 27303 server_base.cc:1034] running on GCE node
I20260927 23:03:37.266063 27303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:37.267649 27303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:37.268934 27303 hybrid_clock.cc:648] HybridClock initialized: now 1790550217268850 us; error 58 us; skew 500 ppm
I20260927 23:03:37.269335 27303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:37 dist-test-slave-5pd3 krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550217, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:37.280378 27303 init.cc:377] Logged in from keytab as kudu/127.4.74.189@KRBTEST.COM (short username kudu)
I20260927 23:03:37.282407 27303 webserver.cc:460] Webserver started at http://127.4.74.189:44973/ using document root <none> and password file <none>
I20260927 23:03:37.283234 27303 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:37.283391 27303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:37.283825 27303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:37.287178 27303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/data/instance:
uuid: "449a34b1422541f0b53af67af0130286"
format_stamp: "Formatted at 2026-09-27 23:03:37 on dist-test-slave-5pd3"
server_key: "91f62684df9bf8017c6c966e2138ab63"
server_key_iv: "5677dc97a2a7f69f97f1f02335ac01c8"
server_key_version: "encryptionkey@0"
I20260927 23:03:37.288137 27303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/wal/instance:
uuid: "449a34b1422541f0b53af67af0130286"
format_stamp: "Formatted at 2026-09-27 23:03:37 on dist-test-slave-5pd3"
server_key: "91f62684df9bf8017c6c966e2138ab63"
server_key_iv: "5677dc97a2a7f69f97f1f02335ac01c8"
server_key_version: "encryptionkey@0"
I20260927 23:03:37.294669 27303 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260927 23:03:37.299784 27318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:37.301666 27303 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260927 23:03:37.301942 27303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/wal
uuid: "449a34b1422541f0b53af67af0130286"
format_stamp: "Formatted at 2026-09-27 23:03:37 on dist-test-slave-5pd3"
server_key: "91f62684df9bf8017c6c966e2138ab63"
server_key_iv: "5677dc97a2a7f69f97f1f02335ac01c8"
server_key_version: "encryptionkey@0"
I20260927 23:03:37.302227 27303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:37.335368 27303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:37.340354 27303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:37.340664 27303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:37.362469 27303 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.189:41733
I20260927 23:03:37.362507 27369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.189:41733 every 8 connection(s)
I20260927 23:03:37.363991 27303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/data/info.pb
I20260927 23:03:37.368778 27370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:37.371420 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 27303
I20260927 23:03:37.371719 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-1/wal/instance
I20260927 23:03:37.372351 4394 external_mini_cluster.cc:1428] Setting key bbdc0caef5b1d22b5646bc440b128149
I20260927 23:03:37.380472 27370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:03:37 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550217, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.74.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.74.188@KRBTEST.COM" created.
I20260927 23:03:37.413658 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.385016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49778 (local address 127.4.74.190:36233)
0927 23:03:37.385749 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:37.385759 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:37.388642 (+ 2883us) server_negotiation.cc:407] Connection header received
0927 23:03:37.392565 (+ 3923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:37.392570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:37.392699 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:37.392851 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:37.396102 (+ 3251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.396986 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.399849 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.400394 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.400528 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:37.405829 (+ 5301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:37.405836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:37.405840 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:37.405896 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:37.409481 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:37.410592 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:37.410598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:37.410601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:37.410739 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:37.411223 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:37.411233 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:37.411235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:37.411629 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:37.411808 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:37.413091 (+ 1283us) server_negotiation.cc:299] Negotiation successful
0927 23:03:37.413356 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":345,"threads_started":1}
I20260927 23:03:37.415459 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.386878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36233 (local address 127.0.0.1:49778)
0927 23:03:37.388305 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.388409 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:37.392028 (+ 3619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:37.393217 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:37.393235 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:37.394188 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:37.395539 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:37.395625 (+ 86us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.397248 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.397257 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:37.399516 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:37.399581 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.400706 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.400716 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:37.401106 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:37.402019 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:37.402060 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:37.405457 (+ 3397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:37.409708 (+ 4251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:37.409727 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:37.409746 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:37.410424 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:37.410864 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:37.410872 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:37.410876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:37.411092 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:37.411781 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:37.411791 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:37.412387 (+ 596us) client_negotiation.cc:769] Sending connection context
0927 23:03:37.412815 (+ 428us) client_negotiation.cc:240] Negotiation successful
0927 23:03:37.413568 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"thread_start_us":536,"threads_started":1}
I20260927 23:03:37.418045 27370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:03:37 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550217, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
I20260927 23:03:37.439121 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.419221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42108)
0927 23:03:37.419506 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.419532 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:37.419719 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:37.420795 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:37.420804 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:37.421416 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:37.421684 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:37.421700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.423261 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.423270 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:37.424687 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:37.424701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.426300 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.426306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:37.426464 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:37.427344 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:37.427366 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:37.430045 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:37.435623 (+ 5578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:37.435629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:37.435633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:37.436142 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:37.436756 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:37.436759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:37.436762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:37.436920 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:37.437842 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:37.437846 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:37.438120 (+ 274us) client_negotiation.cc:769] Sending connection context
0927 23:03:37.438313 (+ 193us) client_negotiation.cc:240] Negotiation successful
0927 23:03:37.438673 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260927 23:03:37.439518 27375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.419400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42108 (local address 127.4.74.189:41733)
0927 23:03:37.420254 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:37.420261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:37.420295 (+ 34us) server_negotiation.cc:407] Connection header received
0927 23:03:37.420360 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:37.420367 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:37.420587 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:37.420733 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:37.421921 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.423088 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.424905 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.426046 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.426231 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:37.430439 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:37.430463 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:37.430488 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:37.430550 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:37.435328 (+ 4778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:37.436396 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:37.436410 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:37.436418 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:37.436622 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:37.437068 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:37.437077 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:37.437083 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:37.437665 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:37.437940 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:37.438848 (+ 908us) server_negotiation.cc:299] Negotiation successful
0927 23:03:37.439153 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":519,"threads_started":1}
I20260927 23:03:37.440618 27296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } attempt: 5
I20260927 23:03:37.442863 27370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:03:37.444837 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.443988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:41976)
0927 23:03:37.444345 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.444553 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
W20260927 23:03:37.445127 27320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
I20260927 23:03:37.447504 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.446951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:41980)
0927 23:03:37.447169 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.447326 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260927 23:03:37.447911 27370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:42203: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Sep 27 23:03:37 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550216, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.74.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab.
Entry for principal kudu/127.4.74.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab.
I20260927 23:03:37.457759 4394 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEVsig8/build/asan/bin/kudu
/tmp/dist-test-taskEVsig8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.188:42203
--webserver_interface=127.4.74.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab
--principal=kudu/127.4.74.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.74.148:35105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.74.190:36233,127.4.74.189:41733,127.4.74.188:42203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 23:03:37.458233 27375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.441916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42120 (local address 127.4.74.189:41733)
0927 23:03:37.442100 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:37.442104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:37.442118 (+ 14us) server_negotiation.cc:407] Connection header received
0927 23:03:37.442164 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:37.442167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:37.442391 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:37.443286 (+ 895us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:37.443732 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.444608 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.446395 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.447026 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.447174 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:37.451268 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:37.451275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:37.451278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:37.451322 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:37.454541 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:37.455407 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:37.455411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:37.455414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:37.455643 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:37.456141 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:37.456148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:37.456152 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:37.456644 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:37.456866 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:37.457519 (+ 653us) server_negotiation.cc:299] Negotiation successful
0927 23:03:37.457809 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 23:03:37.458245 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.441531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42120)
0927 23:03:37.441753 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.441778 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:37.441917 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:37.442584 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:37.442589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:37.443215 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:37.443566 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:37.443581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.444810 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.444817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:37.446146 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:37.446167 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:37.447288 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:37.447295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:37.447613 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:37.448488 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:37.448509 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:37.450991 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:37.454764 (+ 3773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:37.454769 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:37.454772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:37.455274 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:37.455772 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:37.455775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:37.455778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:37.455966 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:37.456842 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:37.456850 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:37.457130 (+ 280us) client_negotiation.cc:769] Sending connection context
0927 23:03:37.457361 (+ 231us) client_negotiation.cc:240] Negotiation successful
0927 23:03:37.457722 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260927 23:03:37.459908 27296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:03:37.461393 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.460757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:41988)
0927 23:03:37.461027 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.461156 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260927 23:03:37.462774 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.462301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:41996)
0927 23:03:37.462502 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.462619 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260927 23:03:37.463069 27296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:42203: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
I20260927 23:03:37.497331 27370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } attempt: 1
I20260927 23:03:37.498961 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.498322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42004)
0927 23:03:37.498608 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.498748 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260927 23:03:37.500272 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.499876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42012)
0927 23:03:37.500062 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.500153 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260927 23:03:37.500506 27370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:42203: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
I20260927 23:03:37.514362 27296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } attempt: 1
I20260927 23:03:37.515672 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.515149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42016)
0927 23:03:37.515344 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.515442 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260927 23:03:37.516641 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.516317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42026)
0927 23:03:37.516468 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.516545 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260927 23:03:37.516847 27296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:42203: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
I20260927 23:03:37.568754 27370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } attempt: 2
I20260927 23:03:37.570255 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.569714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42034)
0927 23:03:37.569982 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.570094 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260927 23:03:37.571393 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.571069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42044)
0927 23:03:37.571209 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.571285 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260927 23:03:37.571619 27370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:42203: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
I20260927 23:03:37.585060 27296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } attempt: 2
I20260927 23:03:37.586220 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.585738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42050)
0927 23:03:37.585974 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.586066 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260927 23:03:37.587136 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.586820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42066)
0927 23:03:37.586977 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.587046 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260927 23:03:37.587359 27296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:42203: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
I20260927 23:03:37.662909 27370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } attempt: 3
I20260927 23:03:37.664597 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.663931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42078)
0927 23:03:37.664267 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.664417 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260927 23:03:37.666085 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.665585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42094)
0927 23:03:37.665883 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.665964 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260927 23:03:37.666320 27370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:42203: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
I20260927 23:03:37.693681 27296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } attempt: 3
I20260927 23:03:37.695317 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.694656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42110)
0927 23:03:37.694968 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.695086 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260927 23:03:37.696813 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.696354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42122)
0927 23:03:37.696602 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.696691 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260927 23:03:37.697054 27296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:42203: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
I20260927 23:03:37.809574 27370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } attempt: 4
I20260927 23:03:37.811183 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.810597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42136)
0927 23:03:37.810892 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.811006 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260927 23:03:37.812497 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.812141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42140)
0927 23:03:37.812304 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.812387 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260927 23:03:37.812708 27370 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:42203: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
I20260927 23:03:37.874322 27296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } attempt: 4
I20260927 23:03:37.875707 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.875150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42154)
0927 23:03:37.875375 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.875466 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260927 23:03:37.877133 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:37.876672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42170)
0927 23:03:37.876862 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:37.876956 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260927 23:03:37.877389 27296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.74.188:42203: Network error: Client connection negotiation failed: client connection to 127.4.74.188:42203: connect: Connection refused (error 111)
W20260927 23:03:37.886686 27378 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 23:03:37.887223 27378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:03:37.887387 27378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:03:37.887612 27378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:03:37.898452 27378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:03:37.898666 27378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 23:03:37.898797 27378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:03:37.898897 27378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:03:37.899019 27378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:03:37.911628 27378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.74.148:35105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.74.190:36233,127.4.74.189:41733,127.4.74.188:42203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/krb5kdc/kudu_127.4.74.188.keytab
--principal=kudu/127.4.74.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.74.188:42203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.74.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:00:12 UTC on ccycloud.quasar-sxziyj.root.comops.site
build id 4676390
ASAN enabled
I20260927 23:03:37.913486 27378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:03:37.915449 27378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:03:37.932233 27386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:37.932317 27384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:03:37.932336 27383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:03:37.934492 27378 server_base.cc:1034] running on GCE node
I20260927 23:03:37.935855 27378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:03:37.937996 27378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:03:37.939338 27378 hybrid_clock.cc:648] HybridClock initialized: now 1790550217939251 us; error 59 us; skew 500 ppm
I20260927 23:03:37.939901 27378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 27 23:03:37 dist-test-slave-5pd3 krb5kdc[27216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550217, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 23:03:37.944736 27378 init.cc:377] Logged in from keytab as kudu/127.4.74.188@KRBTEST.COM (short username kudu)
I20260927 23:03:37.946936 27378 webserver.cc:460] Webserver started at http://127.4.74.188:37039/ using document root <none> and password file <none>
I20260927 23:03:37.948001 27378 fs_manager.cc:359] Metadata directory not provided
I20260927 23:03:37.948172 27378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:03:37.948580 27378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:03:37.952586 27378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/data/instance:
uuid: "8fb93f11bfd948899810e70128181cb2"
format_stamp: "Formatted at 2026-09-27 23:03:37 on dist-test-slave-5pd3"
server_key: "83c48b6b69807bae362f906d02e1c8d1"
server_key_iv: "a4e787d14b3cf5e919cc19b1f0ed6e9d"
server_key_version: "encryptionkey@0"
I20260927 23:03:37.953605 27378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/wal/instance:
uuid: "8fb93f11bfd948899810e70128181cb2"
format_stamp: "Formatted at 2026-09-27 23:03:37 on dist-test-slave-5pd3"
server_key: "83c48b6b69807bae362f906d02e1c8d1"
server_key_iv: "a4e787d14b3cf5e919cc19b1f0ed6e9d"
server_key_version: "encryptionkey@0"
I20260927 23:03:37.961087 27378 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260927 23:03:37.967813 27393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:03:37.970103 27378 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260927 23:03:37.970474 27378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/data,/tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/wal
uuid: "8fb93f11bfd948899810e70128181cb2"
format_stamp: "Formatted at 2026-09-27 23:03:37 on dist-test-slave-5pd3"
server_key: "83c48b6b69807bae362f906d02e1c8d1"
server_key_iv: "a4e787d14b3cf5e919cc19b1f0ed6e9d"
server_key_version: "encryptionkey@0"
I20260927 23:03:37.970851 27378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:03:37.989740 27378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:03:37.994994 27378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:03:37.995374 27378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:03:38.020803 27378 rpc_server.cc:307] RPC server started. Bound to: 127.4.74.188:42203
I20260927 23:03:38.020826 27444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.74.188:42203 every 8 connection(s)
I20260927 23:03:38.022780 27378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/data/info.pb
I20260927 23:03:38.023137 4394 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEVsig8/build/asan/bin/kudu as pid 27378
I20260927 23:03:38.023389 4394 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEVsig8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790549821755302-4394-0/minicluster-data/master-2/wal/instance
I20260927 23:03:38.023954 4394 external_mini_cluster.cc:1428] Setting key a9eea14143aa51841c05ba4728cbe2fb
I20260927 23:03:38.029582 27445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:03:38.039899 27445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:03:38 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
I20260927 23:03:38.053314 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.026210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.4.74.188:42203)
0927 23:03:38.027319 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.027339 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.027407 (+ 68us) server_negotiation.cc:407] Connection header received
0927 23:03:38.030114 (+ 2707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.030152 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.030807 (+ 655us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.031362 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.032884 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.035045 (+ 2161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.036620 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.037871 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.038113 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.042630 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.042643 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.042670 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.042736 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.047921 (+ 5185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.048900 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.048910 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.048919 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.049146 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.049586 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.049591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.049595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.050355 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.050592 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.051258 (+ 666us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.051778 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":507,"threads_started":1}
Sep 27 23:03:38 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550217, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
I20260927 23:03:38.063925 27450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.043690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.190:36233 (local address 127.0.0.1:49786)
0927 23:03:38.044802 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:38.044874 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:38.045171 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:38.045758 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:38.045798 (+ 40us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:38.046732 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:38.047083 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:38.047093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.048324 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.048329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:38.049925 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:38.049937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.050880 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.050909 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:38.051120 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.052027 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:38.052054 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:38.055363 (+ 3309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:38.060585 (+ 5222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:38.060596 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:38.060602 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:38.061152 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:38.061709 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:38.061748 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:38.061752 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:38.061989 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:38.062665 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:38.062674 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:38.062912 (+ 238us) client_negotiation.cc:769] Sending connection context
0927 23:03:38.063160 (+ 248us) client_negotiation.cc:240] Negotiation successful
0927 23:03:38.063584 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":477,"threads_started":1}
I20260927 23:03:38.064138 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.043585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.4.74.190:36233)
0927 23:03:38.044261 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.044271 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.045308 (+ 1037us) server_negotiation.cc:407] Connection header received
0927 23:03:38.045392 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.045399 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.045519 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.045637 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.047261 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.048133 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.050094 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.050678 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.050832 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.055700 (+ 4868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.055744 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.055748 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.055801 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.060379 (+ 4578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.061306 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.061314 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.061319 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.061532 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.062144 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.062149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.062176 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.062505 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.062682 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.063287 (+ 605us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.063603 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":320,"threads_started":1}
I20260927 23:03:38.066160 27445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:03:38 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.190@KRBTEST.COM
Sep 27 23:03:38 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550217, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
I20260927 23:03:38.077641 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.057482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49796 (local address 127.4.74.190:36233)
0927 23:03:38.058496 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.058502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.058531 (+ 29us) server_negotiation.cc:407] Connection header received
0927 23:03:38.058630 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.058636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.058756 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.058919 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.060428 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.061807 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.063587 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.064389 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.064561 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.070221 (+ 5660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.070236 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.070242 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.070296 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.073780 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.074884 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.074890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.074893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.075162 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.075724 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.075728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.075730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.076123 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.076510 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.076911 (+ 401us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.077184 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":415,"threads_started":1}
I20260927 23:03:38.084084 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.067265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42126 (local address 127.4.74.189:41733)
0927 23:03:38.067877 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.067884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.067971 (+ 87us) server_negotiation.cc:407] Connection header received
0927 23:03:38.068182 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.068186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.068256 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.068370 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.069216 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.069960 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.071359 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.072166 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.072341 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.076232 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.076241 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.076246 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.076302 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.080277 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.081466 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.081474 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.081480 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.081693 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.082242 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.082248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.082253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.082638 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.082840 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.083460 (+ 620us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.083727 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":377,"threads_started":1}
I20260927 23:03:38.084122 27450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.067050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.189:41733 (local address 127.0.0.1:42126)
0927 23:03:38.067874 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:38.067893 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:38.068047 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:38.068391 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:38.068395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:38.068822 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:38.069034 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:38.069043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.070112 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.070118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:38.071126 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:38.071145 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.072391 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.072400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:38.072626 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.073222 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:38.073243 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:38.075922 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:38.080561 (+ 4639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:38.080568 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:38.080573 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:38.081285 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:38.081921 (+ 636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:38.081926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:38.081930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:38.082122 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:38.082819 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:38.082824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:38.083103 (+ 279us) client_negotiation.cc:769] Sending connection context
0927 23:03:38.083311 (+ 208us) client_negotiation.cc:240] Negotiation successful
0927 23:03:38.083712 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705}
I20260927 23:03:38.085386 27445 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 23:03:38 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.74.189@KRBTEST.COM
Sep 27 23:03:38 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550217, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.188@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
I20260927 23:03:38.098312 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.080063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42128 (local address 127.4.74.189:41733)
0927 23:03:38.080866 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.080872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.080898 (+ 26us) server_negotiation.cc:407] Connection header received
0927 23:03:38.080954 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.080958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.081042 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.081163 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.082972 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.084397 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.086158 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.086737 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.086876 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.091757 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.091765 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.091771 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.091812 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.095033 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.096003 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.096011 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.096018 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.096208 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.096672 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.096679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.096684 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.097043 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.097336 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.097615 (+ 279us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.097860 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":367,"threads_started":1}
I20260927 23:03:38.101326 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.086487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42202 (local address 127.4.74.188:42203)
0927 23:03:38.086697 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.086704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.086725 (+ 21us) server_negotiation.cc:407] Connection header received
0927 23:03:38.086782 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.086789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.086907 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.087014 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.088014 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.088699 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.089835 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.090224 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.090355 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.094375 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.094381 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.094385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.094428 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.097648 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.098518 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.098528 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.098533 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.098740 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.099308 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.099318 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.099323 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.099821 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.100007 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.100542 (+ 535us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.100854 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260927 23:03:38.102371 27450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.086256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42202)
0927 23:03:38.086437 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:38.086458 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:38.086574 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:38.087165 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:38.087168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:38.087702 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:38.087869 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:38.087885 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.088823 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.088831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:38.089680 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:38.089691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.090399 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.090404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:38.090631 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.091238 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:38.091256 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:38.094104 (+ 2848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:38.097794 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:38.097799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:38.097803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:38.098287 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:38.098959 (+ 672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:38.098963 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:38.098966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:38.099141 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:38.100069 (+ 928us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:38.100076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:38.100257 (+ 181us) client_negotiation.cc:769] Sending connection context
0927 23:03:38.101553 (+ 1296us) client_negotiation.cc:240] Negotiation successful
0927 23:03:38.101931 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260927 23:03:38.105805 4394 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 23:03:38.110054 27445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2: Bootstrap starting.
I20260927 23:03:38.111919 27370 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } attempt: 5
I20260927 23:03:38.109477 27271 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d8956e81eccd4cf2925831ac07b1da17"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49796
I20260927 23:03:38.113946 27445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:38.115437 27445 log.cc:824] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:38.119341 27445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2: No bootstrap required, opened a new log
Sep 27 23:03:38 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550217, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.189@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
I20260927 23:03:38.132119 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.112563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42218)
0927 23:03:38.112840 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:38.112864 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:38.112992 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:38.113545 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:38.113551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:38.114118 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:38.114513 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:38.114533 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.116227 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.116235 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:38.117477 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:38.117496 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.118710 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.118716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:38.118985 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.119735 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:38.119780 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:38.123471 (+ 3691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:38.128809 (+ 5338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:38.128815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:38.128818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:38.129336 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:38.129865 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:38.129875 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:38.129879 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:38.130108 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:38.130925 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:38.130930 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:38.131125 (+ 195us) client_negotiation.cc:769] Sending connection context
0927 23:03:38.131301 (+ 176us) client_negotiation.cc:240] Negotiation successful
0927 23:03:38.131675 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260927 23:03:38.132151 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.112895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.4.74.188:42203)
0927 23:03:38.113160 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.113167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.113197 (+ 30us) server_negotiation.cc:407] Connection header received
0927 23:03:38.113264 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.113268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.113344 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.113462 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.114731 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.116038 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.117707 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.118545 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.118683 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.124045 (+ 5362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.124053 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.124063 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.124175 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.128597 (+ 4422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.129500 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.129506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.129510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.129679 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.130244 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.130249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.130255 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.130773 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.130960 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.131451 (+ 491us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.131743 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260927 23:03:38.132177 27445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } }
I20260927 23:03:38.132736 27445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb93f11bfd948899810e70128181cb2, State: Initialized, Role: FOLLOWER
I20260927 23:03:38.133448 27445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } }
I20260927 23:03:38.134742 27455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } } }
I20260927 23:03:38.135244 27455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:38.136396 27445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:38.138350 27370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286: Bootstrap starting.
I20260927 23:03:38.142470 27370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:38.144002 27370 log.cc:824] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:38.147696 27370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286: No bootstrap required, opened a new log
I20260927 23:03:38.154588 27296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } attempt: 5
I20260927 23:03:38.150418 27378 master_runner.cc:186] Error getting master registration for 127.4.74.190:36233: OK, instance_id { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" instance_seqno: 1790550216712122 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 23:03:38.162822 27378 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 27 23:03:38 dist-test-slave-5pd3 krb5kdc[27216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790550216, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.74.190@KRBTEST.COM for kudu/127.4.74.188@KRBTEST.COM
I20260927 23:03:38.168563 27370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } }
I20260927 23:03:38.169196 27370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 449a34b1422541f0b53af67af0130286, State: Initialized, Role: FOLLOWER
I20260927 23:03:38.169948 27370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } }
W20260927 23:03:38.170888 27467 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:03:38.171195 27467 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:03:38.177469 27469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } } }
I20260927 23:03:38.178246 27469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:38.179767 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.155508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.74.188:42203 (local address 127.0.0.1:42232)
0927 23:03:38.155873 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0927 23:03:38.155898 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0927 23:03:38.156045 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:03:38.156689 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 23:03:38.156696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 23:03:38.157306 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0927 23:03:38.157584 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:38.157596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.159714 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.159720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:38.161257 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 23:03:38.161278 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.163250 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.163260 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 23:03:38.163676 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.164836 (+ 1160us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 23:03:38.164873 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0927 23:03:38.170265 (+ 5392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 23:03:38.175875 (+ 5610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:38.175886 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:38.175893 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:38.176826 (+ 933us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:38.177441 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 23:03:38.177449 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 23:03:38.177454 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0927 23:03:38.177722 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 23:03:38.178479 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 23:03:38.178484 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 23:03:38.178714 (+ 230us) client_negotiation.cc:769] Sending connection context
0927 23:03:38.178907 (+ 193us) client_negotiation.cc:240] Negotiation successful
0927 23:03:38.179282 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214}
I20260927 23:03:38.179795 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.155886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42232 (local address 127.4.74.188:42203)
0927 23:03:38.156182 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.156190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.156222 (+ 32us) server_negotiation.cc:407] Connection header received
0927 23:03:38.156319 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.156328 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.156462 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.156664 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.157777 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.159444 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.161517 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.162627 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.162833 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.170967 (+ 8134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.170983 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.170991 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.171069 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.175531 (+ 4462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.177043 (+ 1512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.177050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.177054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.177270 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.177880 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.177886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.177892 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.178320 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.178500 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.179050 (+ 550us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.179327 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260927 23:03:38.180807 27370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:38.187474 27296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17: Bootstrap starting.
I20260927 23:03:38.191965 27296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17: Neither blocks nor log segments found. Creating new log.
I20260927 23:03:38.193696 27296 log.cc:824] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17: Log is configured to *not* fsync() on all Append() calls
I20260927 23:03:38.208112 27296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17: No bootstrap required, opened a new log
I20260927 23:03:38.216946 27296 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } }
I20260927 23:03:38.217603 27296 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8956e81eccd4cf2925831ac07b1da17, State: Initialized, Role: FOLLOWER
I20260927 23:03:38.218248 27296 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } }
W20260927 23:03:38.219359 27481 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:03:38.222033 27481 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:03:38.208084 27303 master_runner.cc:186] Error getting master registration for 127.4.74.190:36233: OK, instance_id { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" instance_seqno: 1790550216712122 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 23:03:38.224416 27303 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 23:03:38.232295 27483 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } } }
I20260927 23:03:38.232797 27483 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:38.233848 27296 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:03:38.234045 27271 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 23:03:38.234357 27271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:38.238961 27271 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } }
I20260927 23:03:38.242471 27345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8956e81eccd4cf2925831ac07b1da17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "449a34b1422541f0b53af67af0130286"
I20260927 23:03:38.243232 27345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:03:38.248242 27399 tablet.cc:2369] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:03:38.247860 27345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8956e81eccd4cf2925831ac07b1da17 in term 1.
I20260927 23:03:38.250571 27248 leader_election.cc:302] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 449a34b1422541f0b53af67af0130286, d8956e81eccd4cf2925831ac07b1da17; no voters:
I20260927 23:03:38.251304 27483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:03:38.252005 27271 leader_election.cc:288] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [CANDIDATE]: Term 1 election: Requested vote from peers 449a34b1422541f0b53af67af0130286 (127.4.74.189:41733), 8fb93f11bfd948899810e70128181cb2 (127.4.74.188:42203)
I20260927 23:03:38.252774 27420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8956e81eccd4cf2925831ac07b1da17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8fb93f11bfd948899810e70128181cb2"
I20260927 23:03:38.253427 27420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:03:38.258754 27420 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8956e81eccd4cf2925831ac07b1da17 in term 1.
I20260927 23:03:38.273378 27483 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [term 1 LEADER]: Becoming Leader. State: Replica: d8956e81eccd4cf2925831ac07b1da17, State: Running, Role: LEADER
I20260927 23:03:38.274503 27483 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } }
I20260927 23:03:38.281796 27487 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8956e81eccd4cf2925831ac07b1da17. Latest consensus state: current_term: 1 leader_uuid: "d8956e81eccd4cf2925831ac07b1da17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } } }
I20260927 23:03:38.282603 27487 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:38.291991 27498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:03:38.311242 27498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:03:38.320240 27505 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 23:03:38.335033 27420 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [term 1 FOLLOWER]: Refusing update from remote peer d8956e81eccd4cf2925831ac07b1da17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:03:38.336625 27345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [term 1 FOLLOWER]: Refusing update from remote peer d8956e81eccd4cf2925831ac07b1da17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:03:38.337697 27483 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:03:38.338721 27483 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:03:38.343844 27324 tablet.cc:2369] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:03:38.364149 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.296631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42140 (local address 127.4.74.189:41733)
0927 23:03:38.296881 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.296889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.303037 (+ 6148us) server_negotiation.cc:407] Connection header received
0927 23:03:38.304662 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.304672 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.304802 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.304898 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.319860 (+ 14962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.320966 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.324447 (+ 3481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.325221 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.325331 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.345903 (+ 20572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.345913 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.345919 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.345985 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.350258 (+ 4273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.353297 (+ 3039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.353303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.353308 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.353547 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.355714 (+ 2167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.355720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.355726 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.356143 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.356580 (+ 437us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.363261 (+ 6681us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.363628 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260927 23:03:38.365911 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.288258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49806 (local address 127.4.74.190:36233)
0927 23:03:38.288573 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.288585 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.302342 (+ 13757us) server_negotiation.cc:407] Connection header received
0927 23:03:38.306486 (+ 4144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.306494 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.306623 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.306726 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.316181 (+ 9455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.317526 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.323215 (+ 5689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.324026 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.324150 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.339772 (+ 15622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.339781 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.339787 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.339843 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.344495 (+ 4652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.352363 (+ 7868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.352371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.352375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.352605 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.354118 (+ 1513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.354124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.354128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.354762 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.355064 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.365168 (+ 10104us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.365510 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260927 23:03:38.372309 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:38.304092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.4.74.188:42203)
0927 23:03:38.304406 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.304415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.312923 (+ 8508us) server_negotiation.cc:407] Connection header received
0927 23:03:38.313481 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.313502 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.313617 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.313704 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.327681 (+ 13977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.328841 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.336009 (+ 7168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:38.336913 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:38.337032 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:38.357201 (+ 20169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 23:03:38.357209 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 23:03:38.357215 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 23:03:38.357270 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0927 23:03:38.361927 (+ 4657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.369201 (+ 7274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.369209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.369213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.369445 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 23:03:38.370092 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 23:03:38.370098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 23:03:38.370101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0927 23:03:38.370543 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 23:03:38.370835 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:38.371522 (+ 687us) server_negotiation.cc:299] Negotiation successful
0927 23:03:38.371870 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260927 23:03:38.384821 27469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8956e81eccd4cf2925831ac07b1da17. Latest consensus state: current_term: 1 leader_uuid: "d8956e81eccd4cf2925831ac07b1da17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } } }
I20260927 23:03:38.385164 27469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:38.392915 27508 mvcc.cc:205] Tried to move back new op lower bound from 7334093694283063296 to 7334093694061363200. Current Snapshot: MvccSnapshot[applied={T|T < 7334093694283063296}]
I20260927 23:03:38.403745 27455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8956e81eccd4cf2925831ac07b1da17. Latest consensus state: current_term: 1 leader_uuid: "d8956e81eccd4cf2925831ac07b1da17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } } }
I20260927 23:03:38.404207 27455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:38.412055 27469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8956e81eccd4cf2925831ac07b1da17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } } }
I20260927 23:03:38.412387 27469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:38.415815 27483 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8956e81eccd4cf2925831ac07b1da17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } } }
I20260927 23:03:38.416167 27483 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:38.416240 27498 catalog_manager.cc:1347] Generated new cluster ID: 752d03f8aac0430da68914a5a61d611e
I20260927 23:03:38.416392 27498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:03:38.416644 27487 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8956e81eccd4cf2925831ac07b1da17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } } }
I20260927 23:03:38.417050 27487 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17 [sys.catalog]: This master's current role is: LEADER
I20260927 23:03:38.431851 27455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8956e81eccd4cf2925831ac07b1da17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8956e81eccd4cf2925831ac07b1da17" member_type: VOTER last_known_addr { host: "127.4.74.190" port: 36233 } } peers { permanent_uuid: "449a34b1422541f0b53af67af0130286" member_type: VOTER last_known_addr { host: "127.4.74.189" port: 41733 } } peers { permanent_uuid: "8fb93f11bfd948899810e70128181cb2" member_type: VOTER last_known_addr { host: "127.4.74.188" port: 42203 } } }
I20260927 23:03:38.432191 27455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:03:38.482306 27498 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:03:38.484886 27498 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:03:38.518280 27498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8956e81eccd4cf2925831ac07b1da17: Generated new TSK 0
I20260927 23:03:38.519980 27498 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 23:03:38.654385 27451 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:38.635326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49810 (local address 127.4.74.190:36233)
0927 23:03:38.635539 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.635576 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.649771 (+ 14195us) server_negotiation.cc:407] Connection header received
0927 23:03:38.652188 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.652197 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.652312 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.652398 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.654061 (+ 1663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49810: BlockingRecv error: recv got EOF from 127.0.0.1:49810 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260927 23:03:38.656430 27453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:38.644709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.4.74.189:41733)
0927 23:03:38.645020 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.645028 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.650297 (+ 5269us) server_negotiation.cc:407] Connection header received
0927 23:03:38.652635 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.652642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.652739 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.652818 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.656083 (+ 3265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42148: BlockingRecv error: recv got EOF from 127.0.0.1:42148 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260927 23:03:38.660633 27448 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:38.651060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42250 (local address 127.4.74.188:42203)
0927 23:03:38.651328 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.651337 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.656125 (+ 4788us) server_negotiation.cc:407] Connection header received
0927 23:03:38.658884 (+ 2759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.658891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.659003 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.659087 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.660358 (+ 1271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42250: BlockingRecv error: recv got EOF from 127.0.0.1:42250 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260927 23:03:38.700892 27453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:38.691076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42158 (local address 127.4.74.189:41733)
0927 23:03:38.691339 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.691346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.698394 (+ 7048us) server_negotiation.cc:407] Connection header received
0927 23:03:38.699050 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.699058 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.699277 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.699374 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.700638 (+ 1264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42158: BlockingRecv error: recv got EOF from 127.0.0.1:42158 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260927 23:03:38.745026 27453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:38.735609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42160 (local address 127.4.74.189:41733)
0927 23:03:38.735810 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.735817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.742861 (+ 7044us) server_negotiation.cc:407] Connection header received
0927 23:03:38.743416 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.743422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.743530 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.743629 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.744749 (+ 1120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42160: BlockingRecv error: recv got EOF from 127.0.0.1:42160 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260927 23:03:38.796809 27453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:38.793680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42172 (local address 127.4.74.189:41733)
0927 23:03:38.793992 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.794000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.794580 (+ 580us) server_negotiation.cc:407] Connection header received
0927 23:03:38.794963 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.795007 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.795128 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.795225 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.796472 (+ 1247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42172: BlockingRecv error: recv got EOF from 127.0.0.1:42172 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260927 23:03:38.853313 27453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:38.843399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42176 (local address 127.4.74.189:41733)
0927 23:03:38.843693 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.843701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.850878 (+ 7177us) server_negotiation.cc:407] Connection header received
0927 23:03:38.851502 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.851510 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.851661 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.851756 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.852979 (+ 1223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42176: BlockingRecv error: recv got EOF from 127.0.0.1:42176 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260927 23:03:38.892591 27453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:38.889896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42178 (local address 127.4.74.189:41733)
0927 23:03:38.890197 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.890208 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.890841 (+ 633us) server_negotiation.cc:407] Connection header received
0927 23:03:38.891117 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.891129 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.891249 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.891376 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.892292 (+ 916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42178: BlockingRecv error: recv got EOF from 127.0.0.1:42178 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
W20260927 23:03:38.915235 27453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:38.912624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42182 (local address 127.4.74.189:41733)
0927 23:03:38.912951 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.912962 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.913574 (+ 612us) server_negotiation.cc:407] Connection header received
0927 23:03:38.913835 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.913839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.913924 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.914043 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.914897 (+ 854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42182: BlockingRecv error: recv got EOF from 127.0.0.1:42182 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260927 23:03:38.957264 27453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:38.954048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.4.74.189:41733)
0927 23:03:38.954403 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:38.954416 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:38.955239 (+ 823us) server_negotiation.cc:407] Connection header received
0927 23:03:38.955624 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:38.955630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:38.955753 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:38.955896 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:38.956910 (+ 1014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42186: BlockingRecv error: recv got EOF from 127.0.0.1:42186 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260927 23:03:39.028651 27453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:39.025808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42202 (local address 127.4.74.189:41733)
0927 23:03:39.026135 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:39.026142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:39.026723 (+ 581us) server_negotiation.cc:407] Connection header received
0927 23:03:39.026986 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:39.026991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:39.027077 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:39.027186 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:39.028381 (+ 1195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42202: BlockingRecv error: recv got EOF from 127.0.0.1:42202 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
W20260927 23:03:39.163621 27453 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:03:39.161324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42210 (local address 127.4.74.189:41733)
0927 23:03:39.161531 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:39.161537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:39.161924 (+ 387us) server_negotiation.cc:407] Connection header received
0927 23:03:39.162136 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:39.162141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:39.162237 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:39.162372 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0927 23:03:39.163327 (+ 955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42210: BlockingRecv error: recv got EOF from 127.0.0.1:42210 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260927 23:03:39.174439 27467 catalog_manager.cc:1259] Loaded cluster ID: 752d03f8aac0430da68914a5a61d611e
I20260927 23:03:39.174557 27467 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2: loading cluster ID for follower catalog manager: success
I20260927 23:03:39.177431 27467 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2: acquiring CA information for follower catalog manager: success
I20260927 23:03:39.179514 27467 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8fb93f11bfd948899810e70128181cb2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:03:39.224844 27481 catalog_manager.cc:1259] Loaded cluster ID: 752d03f8aac0430da68914a5a61d611e
I20260927 23:03:39.224990 27481 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286: loading cluster ID for follower catalog manager: success
I20260927 23:03:39.228060 27481 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286: acquiring CA information for follower catalog manager: success
I20260927 23:03:39.230348 27481 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 449a34b1422541f0b53af67af0130286: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:03:39.432590 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:39.425636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.4.74.189:41733)
0927 23:03:39.425926 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:39.425936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:39.426214 (+ 278us) server_negotiation.cc:407] Connection header received
0927 23:03:39.426363 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:39.426368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:39.426581 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:39.426722 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:03:39.427225 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:39.428132 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:39.429646 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:39.430118 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:39.430242 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:39.430874 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:03:39.431736 (+ 862us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:03:39.431881 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:39.432049 (+ 168us) server_negotiation.cc:299] Negotiation successful
0927 23:03:39.432169 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260927 23:03:39.448101 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:39.440719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42256 (local address 127.4.74.188:42203)
0927 23:03:39.441458 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:39.441462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:39.441483 (+ 21us) server_negotiation.cc:407] Connection header received
0927 23:03:39.441584 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:39.441588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:39.441760 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:39.441926 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:03:39.442593 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:39.443326 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:39.445167 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:39.445716 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:39.445858 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:39.446564 (+ 706us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:03:39.447182 (+ 618us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:03:39.447317 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:39.447476 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 23:03:39.447639 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":491,"threads_started":1}
I20260927 23:03:39.469401 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:39.455131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.4.74.189:41733)
0927 23:03:39.455382 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:39.455391 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:39.461699 (+ 6308us) server_negotiation.cc:407] Connection header received
0927 23:03:39.461857 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:39.461864 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:39.461982 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:39.462096 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:03:39.463418 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:39.464563 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:39.466874 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:39.467522 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:39.467661 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:39.468492 (+ 831us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:03:39.468657 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:03:39.468717 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:39.468976 (+ 259us) server_negotiation.cc:299] Negotiation successful
0927 23:03:39.469065 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260927 23:03:39.470283 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:39.462195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42264 (local address 127.4.74.188:42203)
0927 23:03:39.462429 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:39.462436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:39.463921 (+ 1485us) server_negotiation.cc:407] Connection header received
0927 23:03:39.464095 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:39.464101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:39.464207 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:39.464331 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:03:39.465018 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:39.466070 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:39.467919 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:39.468572 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:39.468717 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:39.469294 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:03:39.469541 (+ 247us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:03:39.469615 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:39.469893 (+ 278us) server_negotiation.cc:299] Negotiation successful
0927 23:03:39.470001 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260927 23:03:39.469903 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 23:03:39.455060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.4.74.190:36233)
0927 23:03:39.460407 (+ 5347us) server_negotiation.cc:206] Beginning negotiation
0927 23:03:39.460419 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 23:03:39.461327 (+ 908us) server_negotiation.cc:407] Connection header received
0927 23:03:39.461780 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 23:03:39.461790 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 23:03:39.462089 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 23:03:39.462321 (+ 232us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 23:03:39.463424 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:39.464734 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:39.466763 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 23:03:39.467434 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 23:03:39.467616 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 23:03:39.468381 (+ 765us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 23:03:39.468991 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 23:03:39.469122 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0927 23:03:39.469350 (+ 228us) server_negotiation.cc:299] Negotiation successful
0927 23:03:39.469466 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5012,"thread_start_us":4758,"threads_started":1}
I20260927 23:03:39.477536 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 27229
I20260927 23:03:39.478765 27291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:03:39.610108 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 27229
W20260927 23:03:39.634486 27320 connection.cc:535] server connection from 127.0.0.1:42120 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 23:03:39.634958 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 27303
I20260927 23:03:39.636432 27365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:03:39.752542 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 27303
I20260927 23:03:39.776755 4394 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskEVsig8/build/asan/bin/kudu pid 27378
I20260927 23:03:39.778159 27440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260927 23:03:39.904903 4394 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEVsig8/build/asan/bin/kudu with pid 27378
2026-09-27T23:03:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5204 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10386 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (398109 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST